Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/with_whim_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/with_whim_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=0),
 'dropout': 0.2,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/train_whim_mean.npz',
                   '/home/saxelrod/chemprop_cov_2/features/whim/train_whim_std.npz'],
 'features_scaling': True,
 'features_size': None,
 'ffn_hidden_size': 1300,
 'ffn_num_layers': 1,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 1300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': False,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/with_whim_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/test_whim_mean.npz',
                                 '/home/saxelrod/chemprop_cov_2/features/whim/test_whim_std.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/val_whim_mean.npz',
                                '/home/saxelrod/chemprop_cov_2/features/whim/val_whim_std.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.455869
Epoch 1
Validation auc = 0.492488
Epoch 2
Validation auc = 0.509390
Epoch 3
Validation auc = 0.582629
Epoch 4
Loss = 6.7299e-01, PNorm = 54.1562, GNorm = 2.0339, lr_0 = 6.5000e-04
Validation auc = 0.598122
Epoch 5
Validation auc = 0.628638
Epoch 6
Validation auc = 0.651643
Epoch 7
Validation auc = 0.642723
Epoch 8
Validation auc = 0.653052
Epoch 9
Loss = 5.5699e-01, PNorm = 54.3109, GNorm = 1.1722, lr_0 = 9.9743e-04
Validation auc = 0.680282
Epoch 10
Validation auc = 0.678873
Epoch 11
Validation auc = 0.698592
Epoch 12
Validation auc = 0.684038
Epoch 13
Validation auc = 0.679812
Epoch 14
Loss = 5.5718e-01, PNorm = 54.5015, GNorm = 1.6032, lr_0 = 9.8890e-04
Validation auc = 0.693427
Epoch 15
Validation auc = 0.708920
Epoch 16
Validation auc = 0.706573
Epoch 17
Validation auc = 0.712207
Epoch 18
Validation auc = 0.696244
Epoch 19
Loss = 4.3700e-01, PNorm = 54.6892, GNorm = 1.1174, lr_0 = 9.8045e-04
Validation auc = 0.714085
Epoch 20
Validation auc = 0.691549
Epoch 21
Validation auc = 0.730516
Epoch 22
Validation auc = 0.697653
Epoch 23
Validation auc = 0.701878
Epoch 24
Loss = 4.3906e-01, PNorm = 54.8597, GNorm = 0.9245, lr_0 = 9.7207e-04
Validation auc = 0.708451
Epoch 25
Validation auc = 0.709859
Epoch 26
Validation auc = 0.709859
Epoch 27
Validation auc = 0.714554
Epoch 28
Validation auc = 0.730516
Epoch 29
Loss = 4.5145e-01, PNorm = 55.0730, GNorm = 3.5087, lr_0 = 9.6376e-04
Validation auc = 0.728638
Epoch 30
Validation auc = 0.722066
Epoch 31
Validation auc = 0.734742
Epoch 32
Validation auc = 0.721127
Epoch 33
Validation auc = 0.706573
Epoch 34
Loss = 3.6525e-01, PNorm = 55.2827, GNorm = 1.0687, lr_0 = 9.5552e-04
Validation auc = 0.706573
Epoch 35
Validation auc = 0.714554
Epoch 36
Validation auc = 0.715493
Epoch 37
Validation auc = 0.732394
Epoch 38
Validation auc = 0.740376
Epoch 39
Loss = 3.3977e-01, PNorm = 55.5028, GNorm = 1.3645, lr_0 = 9.4735e-04
Validation auc = 0.729577
Epoch 40
Validation auc = 0.717371
Epoch 41
Validation auc = 0.722535
Epoch 42
Validation auc = 0.715023
Epoch 43
Validation auc = 0.722066
Epoch 44
Loss = 2.7085e-01, PNorm = 55.7347, GNorm = 1.0096, lr_0 = 9.3925e-04
Validation auc = 0.706103
Epoch 45
Validation auc = 0.715493
Epoch 46
Validation auc = 0.700469
Epoch 47
Validation auc = 0.706573
Epoch 48
Validation auc = 0.689202
Epoch 49
Loss = 2.5635e-01, PNorm = 56.0031, GNorm = 1.4877, lr_0 = 9.3122e-04
Validation auc = 0.659155
Epoch 50
Validation auc = 0.653991
Epoch 51
Validation auc = 0.662441
Epoch 52
Validation auc = 0.678873
Epoch 53
Validation auc = 0.701408
Epoch 54
Loss = 2.8811e-01, PNorm = 56.2905, GNorm = 1.6335, lr_0 = 9.2326e-04
Validation auc = 0.715023
Epoch 55
Validation auc = 0.702347
Epoch 56
Validation auc = 0.719249
Epoch 57
Validation auc = 0.714085
Epoch 58
Validation auc = 0.703756
Epoch 59
Loss = 2.4521e-01, PNorm = 56.5725, GNorm = 0.9670, lr_0 = 9.1537e-04
Validation auc = 0.691080
Epoch 60
Validation auc = 0.683099
Epoch 61
Validation auc = 0.686385
Epoch 62
Validation auc = 0.696244
Epoch 63
Validation auc = 0.696244
Epoch 64
Loss = 1.4984e-01, PNorm = 56.8590, GNorm = 2.1533, lr_0 = 9.0754e-04
Validation auc = 0.678873
Epoch 65
Validation auc = 0.677465
Epoch 66
Validation auc = 0.691549
Epoch 67
Validation auc = 0.698122
Epoch 68
Validation auc = 0.705164
Epoch 69
Loss = 1.1206e-01, PNorm = 57.1171, GNorm = 0.6312, lr_0 = 8.9978e-04
Validation auc = 0.712207
Epoch 70
Validation auc = 0.691549
Epoch 71
Validation auc = 0.668545
Epoch 72
Validation auc = 0.658216
Epoch 73
Validation auc = 0.655399
Epoch 74
Loss = 7.6592e-02, PNorm = 57.3484, GNorm = 0.4177, lr_0 = 8.9209e-04
Validation auc = 0.657746
Epoch 75
Validation auc = 0.667606
Epoch 76
Validation auc = 0.673239
Epoch 77
Validation auc = 0.679812
Epoch 78
Validation auc = 0.689671
Epoch 79
Loss = 8.0230e-02, PNorm = 57.5575, GNorm = 0.9282, lr_0 = 8.8447e-04
Validation auc = 0.681690
Epoch 80
Validation auc = 0.678873
Epoch 81
Validation auc = 0.682160
Epoch 82
Validation auc = 0.690610
Epoch 83
Validation auc = 0.697183
Epoch 84
Loss = 2.8607e-02, PNorm = 57.7204, GNorm = 0.3186, lr_0 = 8.7691e-04
Validation auc = 0.706103
Epoch 85
Validation auc = 0.713615
Epoch 86
Validation auc = 0.715493
Epoch 87
Validation auc = 0.707512
Epoch 88
Validation auc = 0.716901
Epoch 89
Loss = 1.0613e-01, PNorm = 57.8813, GNorm = 0.6627, lr_0 = 8.6941e-04
Validation auc = 0.714085
Epoch 90
Validation auc = 0.713146
Epoch 91
Validation auc = 0.707981
Epoch 92
Validation auc = 0.713615
Epoch 93
Validation auc = 0.723005
Epoch 94
Loss = 5.7916e-02, PNorm = 58.0294, GNorm = 0.5099, lr_0 = 8.6198e-04
Validation auc = 0.739437
Epoch 95
Validation auc = 0.741784
Epoch 96
Validation auc = 0.729577
Epoch 97
Validation auc = 0.692019
Epoch 98
Validation auc = 0.672300
Epoch 99
Loss = 4.2381e-02, PNorm = 58.1907, GNorm = 0.3050, lr_0 = 8.5461e-04
Validation auc = 0.670423
Epoch 100
Validation auc = 0.674648
Epoch 101
Validation auc = 0.685915
Epoch 102
Validation auc = 0.700939
Epoch 103
Validation auc = 0.683568
Epoch 104
Loss = 4.9200e-02, PNorm = 58.3649, GNorm = 0.7123, lr_0 = 8.4730e-04
Validation auc = 0.701408
Epoch 105
Validation auc = 0.708920
Epoch 106
Validation auc = 0.705164
Epoch 107
Validation auc = 0.716901
Epoch 108
Validation auc = 0.737559
Epoch 109
Loss = 1.4823e-01, PNorm = 58.5158, GNorm = 1.3613, lr_0 = 8.4006e-04
Validation auc = 0.715962
Epoch 110
Validation auc = 0.705634
Epoch 111
Validation auc = 0.696714
Epoch 112
Validation auc = 0.702817
Epoch 113
Validation auc = 0.703756
Epoch 114
Loss = 7.7427e-02, PNorm = 58.6746, GNorm = 0.6652, lr_0 = 8.3288e-04
Validation auc = 0.707042
Epoch 115
Validation auc = 0.714554
Epoch 116
Validation auc = 0.710329
Epoch 117
Validation auc = 0.699061
Epoch 118
Validation auc = 0.699531
Epoch 119
Loss = 4.4754e-02, PNorm = 58.8299, GNorm = 0.5610, lr_0 = 8.2576e-04
Validation auc = 0.703756
Epoch 120
Validation auc = 0.685915
Epoch 121
Validation auc = 0.675587
Epoch 122
Validation auc = 0.684507
Epoch 123
Validation auc = 0.707042
Epoch 124
Loss = 7.4345e-02, PNorm = 58.9799, GNorm = 0.9575, lr_0 = 8.1870e-04
Validation auc = 0.708920
Epoch 125
Validation auc = 0.702817
Epoch 126
Validation auc = 0.686385
Epoch 127
Validation auc = 0.678873
Epoch 128
Validation auc = 0.684977
Epoch 129
Loss = 2.6686e-02, PNorm = 59.1238, GNorm = 0.4679, lr_0 = 8.1170e-04
Validation auc = 0.692488
Epoch 130
Validation auc = 0.707512
Epoch 131
Validation auc = 0.713146
Epoch 132
Validation auc = 0.705634
Epoch 133
Validation auc = 0.695305
Epoch 134
Loss = 1.7351e-02, PNorm = 59.2467, GNorm = 0.4180, lr_0 = 8.0476e-04
Validation auc = 0.692019
Epoch 135
Validation auc = 0.685446
Epoch 136
Validation auc = 0.682160
Epoch 137
Validation auc = 0.693427
Epoch 138
Validation auc = 0.689202
Epoch 139
Loss = 9.4372e-03, PNorm = 59.3582, GNorm = 0.2685, lr_0 = 7.9788e-04
Validation auc = 0.681690
Epoch 140
Validation auc = 0.686854
Epoch 141
Validation auc = 0.695775
Epoch 142
Validation auc = 0.686854
Epoch 143
Validation auc = 0.684507
Epoch 144
Loss = 3.6502e-02, PNorm = 59.4751, GNorm = 1.2589, lr_0 = 7.9106e-04
Validation auc = 0.692488
Epoch 145
Validation auc = 0.700939
Epoch 146
Validation auc = 0.701408
Epoch 147
Validation auc = 0.696714
Epoch 148
Validation auc = 0.689671
Epoch 149
Loss = 1.6967e-02, PNorm = 59.5713, GNorm = 0.6103, lr_0 = 7.8430e-04
Validation auc = 0.682629
Epoch 150
Validation auc = 0.683099
Epoch 151
Validation auc = 0.684507
Epoch 152
Validation auc = 0.677465
Epoch 153
Validation auc = 0.669014
Epoch 154
Loss = 2.7601e-02, PNorm = 59.6521, GNorm = 0.4357, lr_0 = 7.7759e-04
Validation auc = 0.682160
Epoch 155
Validation auc = 0.688732
Epoch 156
Validation auc = 0.699061
Epoch 157
Validation auc = 0.708920
Epoch 158
Validation auc = 0.711737
Epoch 159
Loss = 1.5998e-02, PNorm = 59.7359, GNorm = 0.4898, lr_0 = 7.7095e-04
Validation auc = 0.715962
Epoch 160
Validation auc = 0.715962
Epoch 161
Validation auc = 0.714085
Epoch 162
Validation auc = 0.711737
Epoch 163
Validation auc = 0.710329
Epoch 164
Loss = 1.7897e-02, PNorm = 59.8129, GNorm = 0.5480, lr_0 = 7.6436e-04
Validation auc = 0.709859
Epoch 165
Validation auc = 0.709859
Epoch 166
Validation auc = 0.705634
Epoch 167
Validation auc = 0.699531
Epoch 168
Validation auc = 0.702347
Epoch 169
Loss = 1.0615e-02, PNorm = 59.8920, GNorm = 0.2911, lr_0 = 7.5782e-04
Validation auc = 0.721127
Epoch 170
Validation auc = 0.723944
Epoch 171
Validation auc = 0.728638
Epoch 172
Validation auc = 0.735211
Epoch 173
Validation auc = 0.720657
Epoch 174
Loss = 2.1790e-02, PNorm = 59.9927, GNorm = 0.1178, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.704225
Epoch 176
Validation auc = 0.701408
Epoch 177
Validation auc = 0.713615
Epoch 178
Validation auc = 0.721127
Epoch 179
Loss = 5.7855e-03, PNorm = 60.1143, GNorm = 0.2021, lr_0 = 7.4492e-04
Validation auc = 0.720188
Epoch 180
Validation auc = 0.715962
Epoch 181
Validation auc = 0.698592
Epoch 182
Validation auc = 0.695775
Epoch 183
Validation auc = 0.708920
Epoch 184
Loss = 9.1844e-03, PNorm = 60.2148, GNorm = 0.6630, lr_0 = 7.3855e-04
Validation auc = 0.714554
Epoch 185
Validation auc = 0.722066
Epoch 186
Validation auc = 0.717371
Epoch 187
Validation auc = 0.710329
Epoch 188
Validation auc = 0.703286
Epoch 189
Loss = 3.3187e-02, PNorm = 60.3110, GNorm = 0.8927, lr_0 = 7.3224e-04
Validation auc = 0.704695
Epoch 190
Validation auc = 0.714085
Epoch 191
Validation auc = 0.712676
Epoch 192
Validation auc = 0.707981
Epoch 193
Validation auc = 0.712676
Epoch 194
Loss = 9.7386e-03, PNorm = 60.4003, GNorm = 0.3601, lr_0 = 7.2598e-04
Validation auc = 0.714554
Epoch 195
Validation auc = 0.716901
Epoch 196
Validation auc = 0.711268
Epoch 197
Validation auc = 0.721596
Epoch 198
Validation auc = 0.722066
Epoch 199
Loss = 3.2631e-02, PNorm = 60.4809, GNorm = 1.3974, lr_0 = 7.1977e-04
Validation auc = 0.711737
Epoch 200
Validation auc = 0.710329
Epoch 201
Validation auc = 0.720188
Epoch 202
Validation auc = 0.716901
Epoch 203
Validation auc = 0.715493
Epoch 204
Loss = 1.3712e-02, PNorm = 60.5993, GNorm = 0.7750, lr_0 = 7.1362e-04
Validation auc = 0.718310
Epoch 205
Validation auc = 0.714554
Epoch 206
Validation auc = 0.730986
Epoch 207
Validation auc = 0.734742
Epoch 208
Validation auc = 0.727230
Epoch 209
Loss = 9.1868e-02, PNorm = 60.7320, GNorm = 1.2256, lr_0 = 7.0752e-04
Validation auc = 0.720657
Epoch 210
Validation auc = 0.707981
Epoch 211
Validation auc = 0.678873
Epoch 212
Validation auc = 0.680282
Epoch 213
Validation auc = 0.668075
Epoch 214
Loss = 5.4200e-02, PNorm = 60.9114, GNorm = 0.8881, lr_0 = 7.0147e-04
Validation auc = 0.657277
Epoch 215
Validation auc = 0.668075
Epoch 216
Validation auc = 0.670892
Epoch 217
Validation auc = 0.679812
Epoch 218
Validation auc = 0.684977
Epoch 219
Loss = 1.2160e-02, PNorm = 61.0828, GNorm = 0.2386, lr_0 = 6.9548e-04
Validation auc = 0.691549
Epoch 220
Validation auc = 0.692488
Epoch 221
Validation auc = 0.692488
Epoch 222
Validation auc = 0.695775
Epoch 223
Validation auc = 0.692958
Epoch 224
Loss = 1.7256e-02, PNorm = 61.1964, GNorm = 0.6408, lr_0 = 6.8953e-04
Validation auc = 0.695305
Epoch 225
Validation auc = 0.706103
Epoch 226
Validation auc = 0.718310
Epoch 227
Validation auc = 0.712676
Epoch 228
Validation auc = 0.712676
Epoch 229
Loss = 4.8553e-03, PNorm = 61.2870, GNorm = 0.1418, lr_0 = 6.8364e-04
Validation auc = 0.711268
Epoch 230
Validation auc = 0.717371
Epoch 231
Validation auc = 0.722066
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.726291
Epoch 234
Loss = 4.8168e-03, PNorm = 61.3494, GNorm = 0.0667, lr_0 = 6.7779e-04
Validation auc = 0.719249
Epoch 235
Validation auc = 0.715023
Epoch 236
Validation auc = 0.718310
Epoch 237
Validation auc = 0.716432
Epoch 238
Validation auc = 0.719718
Epoch 239
Loss = 5.5010e-02, PNorm = 61.3946, GNorm = 3.6385, lr_0 = 6.7200e-04
Validation auc = 0.713615
Epoch 240
Validation auc = 0.709859
Epoch 241
Validation auc = 0.713615
Epoch 242
Validation auc = 0.708920
Epoch 243
Validation auc = 0.698122
Epoch 244
Loss = 5.0653e-03, PNorm = 61.4577, GNorm = 0.1770, lr_0 = 6.6625e-04
Validation auc = 0.661033
Epoch 245
Validation auc = 0.633803
Epoch 246
Validation auc = 0.629577
Epoch 247
Validation auc = 0.636150
Epoch 248
Validation auc = 0.640845
Epoch 249
Loss = 7.8077e-02, PNorm = 61.5301, GNorm = 1.3195, lr_0 = 6.6056e-04
Validation auc = 0.649765
Epoch 250
Validation auc = 0.664319
Epoch 251
Validation auc = 0.680751
Epoch 252
Validation auc = 0.696714
Epoch 253
Validation auc = 0.704225
Epoch 254
Loss = 2.4462e-02, PNorm = 61.5950, GNorm = 0.9269, lr_0 = 6.5491e-04
Validation auc = 0.707042
Epoch 255
Validation auc = 0.708451
Epoch 256
Validation auc = 0.700939
Epoch 257
Validation auc = 0.692019
Epoch 258
Validation auc = 0.690141
Epoch 259
Loss = 1.3115e-02, PNorm = 61.6547, GNorm = 0.1819, lr_0 = 6.4931e-04
Validation auc = 0.696244
Epoch 260
Validation auc = 0.697653
Epoch 261
Validation auc = 0.698122
Epoch 262
Validation auc = 0.698592
Epoch 263
Validation auc = 0.700000
Epoch 264
Loss = 8.4446e-03, PNorm = 61.6995, GNorm = 0.4743, lr_0 = 6.4376e-04
Validation auc = 0.702347
Epoch 265
Validation auc = 0.700000
Epoch 266
Validation auc = 0.699061
Epoch 267
Validation auc = 0.695775
Epoch 268
Validation auc = 0.692019
Epoch 269
Loss = 9.6414e-04, PNorm = 61.7342, GNorm = 0.0200, lr_0 = 6.3826e-04
Validation auc = 0.689671
Epoch 270
Validation auc = 0.690610
Epoch 271
Validation auc = 0.692019
Epoch 272
Validation auc = 0.696244
Epoch 273
Validation auc = 0.697653
Epoch 274
Loss = 5.4433e-04, PNorm = 61.7569, GNorm = 0.0446, lr_0 = 6.3280e-04
Validation auc = 0.700469
Epoch 275
Validation auc = 0.704695
Epoch 276
Validation auc = 0.704695
Epoch 277
Validation auc = 0.704695
Epoch 278
Validation auc = 0.706103
Epoch 279
Loss = 6.6848e-04, PNorm = 61.7732, GNorm = 0.0227, lr_0 = 6.2739e-04
Validation auc = 0.706573
Epoch 280
Validation auc = 0.706573
Epoch 281
Validation auc = 0.706103
Epoch 282
Validation auc = 0.705634
Epoch 283
Validation auc = 0.705164
Epoch 284
Loss = 2.8083e-04, PNorm = 61.7838, GNorm = 0.0130, lr_0 = 6.2203e-04
Validation auc = 0.704695
Epoch 285
Validation auc = 0.704225
Epoch 286
Validation auc = 0.703286
Epoch 287
Validation auc = 0.703756
Epoch 288
Validation auc = 0.705164
Epoch 289
Loss = 6.3715e-04, PNorm = 61.7943, GNorm = 0.0158, lr_0 = 6.1671e-04
Validation auc = 0.706103
Epoch 290
Validation auc = 0.706103
Epoch 291
Validation auc = 0.705164
Epoch 292
Validation auc = 0.703286
Epoch 293
Validation auc = 0.703756
Epoch 294
Loss = 2.3408e-03, PNorm = 61.8055, GNorm = 0.0074, lr_0 = 6.1144e-04
Validation auc = 0.704695
Epoch 295
Validation auc = 0.706103
Epoch 296
Validation auc = 0.707512
Epoch 297
Validation auc = 0.708920
Epoch 298
Validation auc = 0.709390
Epoch 299
Loss = 7.9169e-04, PNorm = 61.8199, GNorm = 0.0146, lr_0 = 6.0621e-04
Validation auc = 0.707512
Model 0 best validation auc = 0.741784 on epoch 95
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.859121
Ensemble test auc = 0.859121
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.455399
Epoch 1
Validation auc = 0.495305
Epoch 2
Validation auc = 0.502817
Epoch 3
Validation auc = 0.546009
Epoch 4
Loss = 6.3691e-01, PNorm = 54.1514, GNorm = 1.4314, lr_0 = 6.5000e-04
Validation auc = 0.589202
Epoch 5
Validation auc = 0.622535
Epoch 6
Validation auc = 0.648357
Epoch 7
Validation auc = 0.680751
Epoch 8
Validation auc = 0.670892
Epoch 9
Loss = 6.0222e-01, PNorm = 54.3035, GNorm = 2.1753, lr_0 = 9.9743e-04
Validation auc = 0.680282
Epoch 10
Validation auc = 0.663380
Epoch 11
Validation auc = 0.669014
Epoch 12
Validation auc = 0.687793
Epoch 13
Validation auc = 0.693427
Epoch 14
Loss = 5.5470e-01, PNorm = 54.4789, GNorm = 1.3474, lr_0 = 9.8890e-04
Validation auc = 0.700939
Epoch 15
Validation auc = 0.702347
Epoch 16
Validation auc = 0.704225
Epoch 17
Validation auc = 0.693427
Epoch 18
Validation auc = 0.701408
Epoch 19
Loss = 4.6861e-01, PNorm = 54.6612, GNorm = 0.8854, lr_0 = 9.8045e-04
Validation auc = 0.698592
Epoch 20
Validation auc = 0.700469
Epoch 21
Validation auc = 0.708451
Epoch 22
Validation auc = 0.718779
Epoch 23
Validation auc = 0.740376
Epoch 24
Loss = 4.8216e-01, PNorm = 54.8565, GNorm = 2.5290, lr_0 = 9.7207e-04
Validation auc = 0.737089
Epoch 25
Validation auc = 0.744131
Epoch 26
Validation auc = 0.734742
Epoch 27
Validation auc = 0.727230
Epoch 28
Validation auc = 0.718310
Epoch 29
Loss = 4.3704e-01, PNorm = 55.0293, GNorm = 1.3268, lr_0 = 9.6376e-04
Validation auc = 0.712207
Epoch 30
Validation auc = 0.711268
Epoch 31
Validation auc = 0.717840
Epoch 32
Validation auc = 0.727230
Epoch 33
Validation auc = 0.732394
Epoch 34
Loss = 3.4878e-01, PNorm = 55.2445, GNorm = 0.8985, lr_0 = 9.5552e-04
Validation auc = 0.725352
Epoch 35
Validation auc = 0.726761
Epoch 36
Validation auc = 0.729108
Epoch 37
Validation auc = 0.735681
Epoch 38
Validation auc = 0.730047
Epoch 39
Loss = 3.4763e-01, PNorm = 55.4364, GNorm = 1.3128, lr_0 = 9.4735e-04
Validation auc = 0.728638
Epoch 40
Validation auc = 0.713615
Epoch 41
Validation auc = 0.707042
Epoch 42
Validation auc = 0.693897
Epoch 43
Validation auc = 0.703756
Epoch 44
Loss = 3.2660e-01, PNorm = 55.6634, GNorm = 2.5235, lr_0 = 9.3925e-04
Validation auc = 0.715023
Epoch 45
Validation auc = 0.716901
Epoch 46
Validation auc = 0.727230
Epoch 47
Validation auc = 0.732864
Epoch 48
Validation auc = 0.729577
Epoch 49
Loss = 2.2016e-01, PNorm = 55.9187, GNorm = 0.9826, lr_0 = 9.3122e-04
Validation auc = 0.723005
Epoch 50
Validation auc = 0.718779
Epoch 51
Validation auc = 0.705164
Epoch 52
Validation auc = 0.708920
Epoch 53
Validation auc = 0.715962
Epoch 54
Loss = 2.1161e-01, PNorm = 56.1958, GNorm = 1.5132, lr_0 = 9.2326e-04
Validation auc = 0.705634
Epoch 55
Validation auc = 0.704695
Epoch 56
Validation auc = 0.694366
Epoch 57
Validation auc = 0.696244
Epoch 58
Validation auc = 0.721127
Epoch 59
Loss = 1.5458e-01, PNorm = 56.4866, GNorm = 0.8502, lr_0 = 9.1537e-04
Validation auc = 0.706103
Epoch 60
Validation auc = 0.707512
Epoch 61
Validation auc = 0.697183
Epoch 62
Validation auc = 0.700939
Epoch 63
Validation auc = 0.707512
Epoch 64
Loss = 1.2534e-01, PNorm = 56.7311, GNorm = 1.1744, lr_0 = 9.0754e-04
Validation auc = 0.696714
Epoch 65
Validation auc = 0.687793
Epoch 66
Validation auc = 0.675587
Epoch 67
Validation auc = 0.683568
Epoch 68
Validation auc = 0.693897
Epoch 69
Loss = 1.4944e-01, PNorm = 56.9663, GNorm = 1.6711, lr_0 = 8.9978e-04
Validation auc = 0.694836
Epoch 70
Validation auc = 0.697653
Epoch 71
Validation auc = 0.699531
Epoch 72
Validation auc = 0.717371
Epoch 73
Validation auc = 0.718779
Epoch 74
Loss = 1.5657e-01, PNorm = 57.1801, GNorm = 0.5434, lr_0 = 8.9209e-04
Validation auc = 0.708920
Epoch 75
Validation auc = 0.698122
Epoch 76
Validation auc = 0.703286
Epoch 77
Validation auc = 0.693897
Epoch 78
Validation auc = 0.703756
Epoch 79
Loss = 8.0044e-02, PNorm = 57.4082, GNorm = 0.3554, lr_0 = 8.8447e-04
Validation auc = 0.723005
Epoch 80
Validation auc = 0.724883
Epoch 81
Validation auc = 0.713615
Epoch 82
Validation auc = 0.724883
Epoch 83
Validation auc = 0.724883
Epoch 84
Loss = 8.7627e-02, PNorm = 57.6278, GNorm = 1.7698, lr_0 = 8.7691e-04
Validation auc = 0.715493
Epoch 85
Validation auc = 0.729577
Epoch 86
Validation auc = 0.712207
Epoch 87
Validation auc = 0.690141
Epoch 88
Validation auc = 0.690610
Epoch 89
Loss = 1.1119e-01, PNorm = 57.8227, GNorm = 1.4348, lr_0 = 8.6941e-04
Validation auc = 0.691080
Epoch 90
Validation auc = 0.675117
Epoch 91
Validation auc = 0.699061
Epoch 92
Validation auc = 0.708451
Epoch 93
Validation auc = 0.694836
Epoch 94
Loss = 1.5039e-01, PNorm = 58.0166, GNorm = 2.8759, lr_0 = 8.6198e-04
Validation auc = 0.698592
Epoch 95
Validation auc = 0.708451
Epoch 96
Validation auc = 0.697183
Epoch 97
Validation auc = 0.695775
Epoch 98
Validation auc = 0.696714
Epoch 99
Loss = 6.5595e-02, PNorm = 58.2321, GNorm = 0.3032, lr_0 = 8.5461e-04
Validation auc = 0.722535
Epoch 100
Validation auc = 0.721127
Epoch 101
Validation auc = 0.702347
Epoch 102
Validation auc = 0.704695
Epoch 103
Validation auc = 0.699061
Epoch 104
Loss = 1.0154e-01, PNorm = 58.4118, GNorm = 0.8000, lr_0 = 8.4730e-04
Validation auc = 0.692958
Epoch 105
Validation auc = 0.680282
Epoch 106
Validation auc = 0.685446
Epoch 107
Validation auc = 0.690141
Epoch 108
Validation auc = 0.684977
Epoch 109
Loss = 1.6787e-02, PNorm = 58.5625, GNorm = 0.2064, lr_0 = 8.4006e-04
Validation auc = 0.687324
Epoch 110
Validation auc = 0.693897
Epoch 111
Validation auc = 0.707981
Epoch 112
Validation auc = 0.709859
Epoch 113
Validation auc = 0.699061
Epoch 114
Loss = 2.4139e-02, PNorm = 58.6905, GNorm = 0.3087, lr_0 = 8.3288e-04
Validation auc = 0.695305
Epoch 115
Validation auc = 0.690141
Epoch 116
Validation auc = 0.702817
Epoch 117
Validation auc = 0.710798
Epoch 118
Validation auc = 0.701878
Epoch 119
Loss = 4.1192e-02, PNorm = 58.8041, GNorm = 1.9348, lr_0 = 8.2576e-04
Validation auc = 0.718310
Epoch 120
Validation auc = 0.730986
Epoch 121
Validation auc = 0.715023
Epoch 122
Validation auc = 0.724413
Epoch 123
Validation auc = 0.736620
Epoch 124
Loss = 8.2310e-02, PNorm = 58.9224, GNorm = 1.4620, lr_0 = 8.1870e-04
Validation auc = 0.723944
Epoch 125
Validation auc = 0.693427
Epoch 126
Validation auc = 0.685915
Epoch 127
Validation auc = 0.689202
Epoch 128
Validation auc = 0.685915
Epoch 129
Loss = 3.1756e-02, PNorm = 59.0941, GNorm = 0.6710, lr_0 = 8.1170e-04
Validation auc = 0.691080
Epoch 130
Validation auc = 0.697183
Epoch 131
Validation auc = 0.707042
Epoch 132
Validation auc = 0.707042
Epoch 133
Validation auc = 0.713615
Epoch 134
Loss = 4.8646e-02, PNorm = 59.2417, GNorm = 0.3081, lr_0 = 8.0476e-04
Validation auc = 0.705634
Epoch 135
Validation auc = 0.725352
Epoch 136
Validation auc = 0.729577
Epoch 137
Validation auc = 0.723005
Epoch 138
Validation auc = 0.717371
Epoch 139
Loss = 1.5983e-02, PNorm = 59.3735, GNorm = 0.1017, lr_0 = 7.9788e-04
Validation auc = 0.704225
Epoch 140
Validation auc = 0.708920
Epoch 141
Validation auc = 0.710329
Epoch 142
Validation auc = 0.717371
Epoch 143
Validation auc = 0.717840
Epoch 144
Loss = 5.5245e-02, PNorm = 59.4837, GNorm = 1.1967, lr_0 = 7.9106e-04
Validation auc = 0.729108
Epoch 145
Validation auc = 0.728169
Epoch 146
Validation auc = 0.700469
Epoch 147
Validation auc = 0.728638
Epoch 148
Validation auc = 0.723944
Epoch 149
Loss = 1.6105e-02, PNorm = 59.5856, GNorm = 1.0357, lr_0 = 7.8430e-04
Validation auc = 0.710798
Epoch 150
Validation auc = 0.710329
Epoch 151
Validation auc = 0.707042
Epoch 152
Validation auc = 0.692019
Epoch 153
Validation auc = 0.712207
Epoch 154
Loss = 4.4738e-02, PNorm = 59.7089, GNorm = 1.2178, lr_0 = 7.7759e-04
Validation auc = 0.723005
Epoch 155
Validation auc = 0.702347
Epoch 156
Validation auc = 0.691549
Epoch 157
Validation auc = 0.687324
Epoch 158
Validation auc = 0.677465
Epoch 159
Loss = 3.5875e-02, PNorm = 59.8313, GNorm = 0.1109, lr_0 = 7.7095e-04
Validation auc = 0.676056
Epoch 160
Validation auc = 0.676995
Epoch 161
Validation auc = 0.687793
Epoch 162
Validation auc = 0.682160
Epoch 163
Validation auc = 0.661972
Epoch 164
Loss = 1.0415e-01, PNorm = 59.9427, GNorm = 3.3434, lr_0 = 7.6436e-04
Validation auc = 0.663850
Epoch 165
Validation auc = 0.685446
Epoch 166
Validation auc = 0.679343
Epoch 167
Validation auc = 0.668075
Epoch 168
Validation auc = 0.670423
Epoch 169
Loss = 2.4201e-02, PNorm = 60.0550, GNorm = 0.2072, lr_0 = 7.5782e-04
Validation auc = 0.682160
Epoch 170
Validation auc = 0.692019
Epoch 171
Validation auc = 0.689671
Epoch 172
Validation auc = 0.690141
Epoch 173
Validation auc = 0.693427
Epoch 174
Loss = 9.3778e-03, PNorm = 60.1460, GNorm = 0.2052, lr_0 = 7.5134e-04
Validation auc = 0.706573
Epoch 175
Validation auc = 0.711737
Epoch 176
Validation auc = 0.714085
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.710798
Epoch 179
Loss = 7.5879e-03, PNorm = 60.2190, GNorm = 0.4306, lr_0 = 7.4492e-04
Validation auc = 0.710329
Epoch 180
Validation auc = 0.712676
Epoch 181
Validation auc = 0.715493
Epoch 182
Validation auc = 0.715023
Epoch 183
Validation auc = 0.710329
Epoch 184
Loss = 1.5681e-02, PNorm = 60.2768, GNorm = 0.7819, lr_0 = 7.3855e-04
Validation auc = 0.705634
Epoch 185
Validation auc = 0.704225
Epoch 186
Validation auc = 0.705634
Epoch 187
Validation auc = 0.707042
Epoch 188
Validation auc = 0.706573
Epoch 189
Loss = 1.0966e-02, PNorm = 60.3262, GNorm = 0.3529, lr_0 = 7.3224e-04
Validation auc = 0.706103
Epoch 190
Validation auc = 0.707042
Epoch 191
Validation auc = 0.704225
Epoch 192
Validation auc = 0.701878
Epoch 193
Validation auc = 0.705634
Epoch 194
Loss = 6.0541e-03, PNorm = 60.3730, GNorm = 0.3001, lr_0 = 7.2598e-04
Validation auc = 0.707512
Epoch 195
Validation auc = 0.709859
Epoch 196
Validation auc = 0.708451
Epoch 197
Validation auc = 0.703286
Epoch 198
Validation auc = 0.710329
Epoch 199
Loss = 1.5631e-03, PNorm = 60.4160, GNorm = 0.0571, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.708451
Epoch 201
Validation auc = 0.701408
Epoch 202
Validation auc = 0.696244
Epoch 203
Validation auc = 0.692488
Epoch 204
Loss = 1.6982e-03, PNorm = 60.4466, GNorm = 0.0177, lr_0 = 7.1362e-04
Validation auc = 0.692488
Epoch 205
Validation auc = 0.691549
Epoch 206
Validation auc = 0.691080
Epoch 207
Validation auc = 0.691315
Epoch 208
Validation auc = 0.693427
Epoch 209
Loss = 2.7914e-03, PNorm = 60.4685, GNorm = 0.2323, lr_0 = 7.0752e-04
Validation auc = 0.693427
Epoch 210
Validation auc = 0.689671
Epoch 211
Validation auc = 0.683568
Epoch 212
Validation auc = 0.682629
Epoch 213
Validation auc = 0.683568
Epoch 214
Loss = 2.3681e-03, PNorm = 60.4925, GNorm = 0.0277, lr_0 = 7.0147e-04
Validation auc = 0.673709
Epoch 215
Validation auc = 0.682160
Epoch 216
Validation auc = 0.696244
Epoch 217
Validation auc = 0.699531
Epoch 218
Validation auc = 0.692958
Epoch 219
Loss = 3.6773e-02, PNorm = 60.5355, GNorm = 0.2117, lr_0 = 6.9548e-04
Validation auc = 0.692958
Epoch 220
Validation auc = 0.703756
Epoch 221
Validation auc = 0.708920
Epoch 222
Validation auc = 0.713146
Epoch 223
Validation auc = 0.713146
Epoch 224
Loss = 8.6403e-02, PNorm = 60.6371, GNorm = 0.0804, lr_0 = 6.8953e-04
Validation auc = 0.722066
Epoch 225
Validation auc = 0.714554
Epoch 226
Validation auc = 0.695775
Epoch 227
Validation auc = 0.694366
Epoch 228
Validation auc = 0.690610
Epoch 229
Loss = 3.6279e-02, PNorm = 60.7797, GNorm = 1.5629, lr_0 = 6.8364e-04
Validation auc = 0.689202
Epoch 230
Validation auc = 0.685446
Epoch 231
Validation auc = 0.681690
Epoch 232
Validation auc = 0.699531
Epoch 233
Validation auc = 0.700939
Epoch 234
Loss = 2.7134e-02, PNorm = 60.9331, GNorm = 0.4675, lr_0 = 6.7779e-04
Validation auc = 0.699531
Epoch 235
Validation auc = 0.694836
Epoch 236
Validation auc = 0.686385
Epoch 237
Validation auc = 0.671831
Epoch 238
Validation auc = 0.665728
Epoch 239
Loss = 1.6657e-02, PNorm = 61.0495, GNorm = 0.6979, lr_0 = 6.7200e-04
Validation auc = 0.666667
Epoch 240
Validation auc = 0.669014
Epoch 241
Validation auc = 0.674178
Epoch 242
Validation auc = 0.675117
Epoch 243
Validation auc = 0.675117
Epoch 244
Loss = 2.3663e-03, PNorm = 61.1295, GNorm = 0.0431, lr_0 = 6.6625e-04
Validation auc = 0.675117
Epoch 245
Validation auc = 0.676056
Epoch 246
Validation auc = 0.676526
Epoch 247
Validation auc = 0.672300
Epoch 248
Validation auc = 0.665728
Epoch 249
Loss = 5.2453e-03, PNorm = 61.1862, GNorm = 0.0411, lr_0 = 6.6056e-04
Validation auc = 0.660094
Epoch 250
Validation auc = 0.653521
Epoch 251
Validation auc = 0.648357
Epoch 252
Validation auc = 0.646948
Epoch 253
Validation auc = 0.648357
Epoch 254
Loss = 1.8923e-03, PNorm = 61.2223, GNorm = 0.0150, lr_0 = 6.5491e-04
Validation auc = 0.655869
Epoch 255
Validation auc = 0.662911
Epoch 256
Validation auc = 0.667606
Epoch 257
Validation auc = 0.669014
Epoch 258
Validation auc = 0.670423
Epoch 259
Loss = 2.4488e-03, PNorm = 61.2466, GNorm = 0.1627, lr_0 = 6.4931e-04
Validation auc = 0.667606
Epoch 260
Validation auc = 0.665728
Epoch 261
Validation auc = 0.666667
Epoch 262
Validation auc = 0.669484
Epoch 263
Validation auc = 0.667136
Epoch 264
Loss = 1.1330e-03, PNorm = 61.2658, GNorm = 0.0097, lr_0 = 6.4376e-04
Validation auc = 0.667136
Epoch 265
Validation auc = 0.667136
Epoch 266
Validation auc = 0.667606
Epoch 267
Validation auc = 0.666667
Epoch 268
Validation auc = 0.667136
Epoch 269
Loss = 1.1446e-03, PNorm = 61.2799, GNorm = 0.0922, lr_0 = 6.3826e-04
Validation auc = 0.667136
Epoch 270
Validation auc = 0.666667
Epoch 271
Validation auc = 0.668075
Epoch 272
Validation auc = 0.667136
Epoch 273
Validation auc = 0.667136
Epoch 274
Loss = 8.3738e-04, PNorm = 61.2903, GNorm = 0.0178, lr_0 = 6.3280e-04
Validation auc = 0.668545
Epoch 275
Validation auc = 0.672300
Epoch 276
Validation auc = 0.672770
Epoch 277
Validation auc = 0.671831
Epoch 278
Validation auc = 0.670423
Epoch 279
Loss = 7.9991e-04, PNorm = 61.3038, GNorm = 0.0285, lr_0 = 6.2739e-04
Validation auc = 0.668545
Epoch 280
Validation auc = 0.667136
Epoch 281
Validation auc = 0.667606
Epoch 282
Validation auc = 0.673239
Epoch 283
Validation auc = 0.677465
Epoch 284
Loss = 3.5256e-04, PNorm = 61.3158, GNorm = 0.0147, lr_0 = 6.2203e-04
Validation auc = 0.678873
Epoch 285
Validation auc = 0.681221
Epoch 286
Validation auc = 0.682160
Epoch 287
Validation auc = 0.682629
Epoch 288
Validation auc = 0.683099
Epoch 289
Loss = 2.2658e-04, PNorm = 61.3244, GNorm = 0.0112, lr_0 = 6.1671e-04
Validation auc = 0.682160
Epoch 290
Validation auc = 0.683099
Epoch 291
Validation auc = 0.684507
Epoch 292
Validation auc = 0.685446
Epoch 293
Validation auc = 0.685915
Epoch 294
Loss = 2.5955e-04, PNorm = 61.3310, GNorm = 0.0050, lr_0 = 6.1144e-04
Validation auc = 0.685446
Epoch 295
Validation auc = 0.684038
Epoch 296
Validation auc = 0.683099
Epoch 297
Validation auc = 0.683099
Epoch 298
Validation auc = 0.684038
Epoch 299
Loss = 6.4434e-04, PNorm = 61.3379, GNorm = 0.0166, lr_0 = 6.0621e-04
Validation auc = 0.684038
Model 0 best validation auc = 0.744131 on epoch 25
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.907887
Ensemble test auc = 0.907887
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.459155
Epoch 1
Validation auc = 0.489671
Epoch 2
Validation auc = 0.521127
Epoch 3
Validation auc = 0.587324
Epoch 4
Loss = 7.2273e-01, PNorm = 54.1630, GNorm = 2.1465, lr_0 = 6.5000e-04
Validation auc = 0.576056
Epoch 5
Validation auc = 0.649296
Epoch 6
Validation auc = 0.643662
Epoch 7
Validation auc = 0.688732
Epoch 8
Validation auc = 0.668075
Epoch 9
Loss = 5.8808e-01, PNorm = 54.3152, GNorm = 1.5245, lr_0 = 9.9743e-04
Validation auc = 0.666197
Epoch 10
Validation auc = 0.690610
Epoch 11
Validation auc = 0.688263
Epoch 12
Validation auc = 0.698592
Epoch 13
Validation auc = 0.723944
Epoch 14
Loss = 5.2696e-01, PNorm = 54.5018, GNorm = 1.0291, lr_0 = 9.8890e-04
Validation auc = 0.693427
Epoch 15
Validation auc = 0.705164
Epoch 16
Validation auc = 0.707042
Epoch 17
Validation auc = 0.701878
Epoch 18
Validation auc = 0.694366
Epoch 19
Loss = 4.7731e-01, PNorm = 54.6831, GNorm = 1.2478, lr_0 = 9.8045e-04
Validation auc = 0.696714
Epoch 20
Validation auc = 0.710798
Epoch 21
Validation auc = 0.698122
Epoch 22
Validation auc = 0.730047
Epoch 23
Validation auc = 0.724413
Epoch 24
Loss = 4.0797e-01, PNorm = 54.8585, GNorm = 0.8731, lr_0 = 9.7207e-04
Validation auc = 0.729108
Epoch 25
Validation auc = 0.735681
Epoch 26
Validation auc = 0.736150
Epoch 27
Validation auc = 0.733803
Epoch 28
Validation auc = 0.739906
Epoch 29
Loss = 5.3154e-01, PNorm = 55.0498, GNorm = 2.5570, lr_0 = 9.6376e-04
Validation auc = 0.742254
Epoch 30
Validation auc = 0.722535
Epoch 31
Validation auc = 0.718779
Epoch 32
Validation auc = 0.726761
Epoch 33
Validation auc = 0.730516
Epoch 34
Loss = 4.0360e-01, PNorm = 55.2355, GNorm = 0.9311, lr_0 = 9.5552e-04
Validation auc = 0.735681
Epoch 35
Validation auc = 0.736620
Epoch 36
Validation auc = 0.744131
Epoch 37
Validation auc = 0.724883
Epoch 38
Validation auc = 0.722535
Epoch 39
Loss = 3.4673e-01, PNorm = 55.4395, GNorm = 1.8418, lr_0 = 9.4735e-04
Validation auc = 0.733803
Epoch 40
Validation auc = 0.736150
Epoch 41
Validation auc = 0.733803
Epoch 42
Validation auc = 0.732864
Epoch 43
Validation auc = 0.738028
Epoch 44
Loss = 3.9743e-01, PNorm = 55.6407, GNorm = 0.5501, lr_0 = 9.3925e-04
Validation auc = 0.723474
Epoch 45
Validation auc = 0.713615
Epoch 46
Validation auc = 0.714554
Epoch 47
Validation auc = 0.718779
Epoch 48
Validation auc = 0.715023
Epoch 49
Loss = 3.0970e-01, PNorm = 55.8784, GNorm = 2.4179, lr_0 = 9.3122e-04
Validation auc = 0.720188
Epoch 50
Validation auc = 0.729108
Epoch 51
Validation auc = 0.715493
Epoch 52
Validation auc = 0.711737
Epoch 53
Validation auc = 0.705634
Epoch 54
Loss = 2.0688e-01, PNorm = 56.1345, GNorm = 0.8780, lr_0 = 9.2326e-04
Validation auc = 0.702817
Epoch 55
Validation auc = 0.706573
Epoch 56
Validation auc = 0.710329
Epoch 57
Validation auc = 0.694836
Epoch 58
Validation auc = 0.693897
Epoch 59
Loss = 2.8421e-01, PNorm = 56.3897, GNorm = 1.8911, lr_0 = 9.1537e-04
Validation auc = 0.690141
Epoch 60
Validation auc = 0.690610
Epoch 61
Validation auc = 0.700469
Epoch 62
Validation auc = 0.707512
Epoch 63
Validation auc = 0.691080
Epoch 64
Loss = 1.8078e-01, PNorm = 56.6726, GNorm = 1.1573, lr_0 = 9.0754e-04
Validation auc = 0.689671
Epoch 65
Validation auc = 0.701878
Epoch 66
Validation auc = 0.710798
Epoch 67
Validation auc = 0.716432
Epoch 68
Validation auc = 0.720657
Epoch 69
Loss = 1.2020e-01, PNorm = 56.9597, GNorm = 1.1341, lr_0 = 8.9978e-04
Validation auc = 0.723005
Epoch 70
Validation auc = 0.714554
Epoch 71
Validation auc = 0.696244
Epoch 72
Validation auc = 0.681690
Epoch 73
Validation auc = 0.688732
Epoch 74
Loss = 9.0995e-02, PNorm = 57.2239, GNorm = 1.3289, lr_0 = 8.9209e-04
Validation auc = 0.693897
Epoch 75
Validation auc = 0.701408
Epoch 76
Validation auc = 0.706573
Epoch 77
Validation auc = 0.680282
Epoch 78
Validation auc = 0.705164
Epoch 79
Loss = 2.4047e-01, PNorm = 57.4563, GNorm = 0.6218, lr_0 = 8.8447e-04
Validation auc = 0.689202
Epoch 80
Validation auc = 0.692019
Epoch 81
Validation auc = 0.701408
Epoch 82
Validation auc = 0.707981
Epoch 83
Validation auc = 0.687324
Epoch 84
Loss = 2.4739e-01, PNorm = 57.7095, GNorm = 1.7878, lr_0 = 8.7691e-04
Validation auc = 0.669484
Epoch 85
Validation auc = 0.688263
Epoch 86
Validation auc = 0.693897
Epoch 87
Validation auc = 0.683568
Epoch 88
Validation auc = 0.681690
Epoch 89
Loss = 2.0915e-01, PNorm = 57.9671, GNorm = 1.3055, lr_0 = 8.6941e-04
Validation auc = 0.690610
Epoch 90
Validation auc = 0.715493
Epoch 91
Validation auc = 0.694836
Epoch 92
Validation auc = 0.682160
Epoch 93
Validation auc = 0.687324
Epoch 94
Loss = 9.0535e-02, PNorm = 58.1860, GNorm = 0.9114, lr_0 = 8.6198e-04
Validation auc = 0.696244
Epoch 95
Validation auc = 0.695305
Epoch 96
Validation auc = 0.691080
Epoch 97
Validation auc = 0.692958
Epoch 98
Validation auc = 0.692958
Epoch 99
Loss = 8.1985e-02, PNorm = 58.3727, GNorm = 0.9856, lr_0 = 8.5461e-04
Validation auc = 0.690610
Epoch 100
Validation auc = 0.688732
Epoch 101
Validation auc = 0.683568
Epoch 102
Validation auc = 0.682629
Epoch 103
Validation auc = 0.700469
Epoch 104
Loss = 5.0923e-02, PNorm = 58.5287, GNorm = 0.3784, lr_0 = 8.4730e-04
Validation auc = 0.691549
Epoch 105
Validation auc = 0.682160
Epoch 106
Validation auc = 0.669484
Epoch 107
Validation auc = 0.671362
Epoch 108
Validation auc = 0.668545
Epoch 109
Loss = 5.9230e-02, PNorm = 58.6634, GNorm = 1.0098, lr_0 = 8.4006e-04
Validation auc = 0.678404
Epoch 110
Validation auc = 0.677934
Epoch 111
Validation auc = 0.669014
Epoch 112
Validation auc = 0.678404
Epoch 113
Validation auc = 0.675587
Epoch 114
Loss = 3.5989e-02, PNorm = 58.8016, GNorm = 1.6240, lr_0 = 8.3288e-04
Validation auc = 0.690610
Epoch 115
Validation auc = 0.708451
Epoch 116
Validation auc = 0.716901
Epoch 117
Validation auc = 0.684507
Epoch 118
Validation auc = 0.629577
Epoch 119
Loss = 2.2507e-01, PNorm = 58.9732, GNorm = 2.1876, lr_0 = 8.2576e-04
Validation auc = 0.625352
Epoch 120
Validation auc = 0.644131
Epoch 121
Validation auc = 0.646948
Epoch 122
Validation auc = 0.649765
Epoch 123
Validation auc = 0.657746
Epoch 124
Loss = 1.2317e-01, PNorm = 59.2760, GNorm = 1.2766, lr_0 = 8.1870e-04
Validation auc = 0.669953
Epoch 125
Validation auc = 0.697183
Epoch 126
Validation auc = 0.690141
Epoch 127
Validation auc = 0.672300
Epoch 128
Validation auc = 0.675117
Epoch 129
Loss = 7.1156e-02, PNorm = 59.5040, GNorm = 0.3141, lr_0 = 8.1170e-04
Validation auc = 0.669953
Epoch 130
Validation auc = 0.670423
Epoch 131
Validation auc = 0.678873
Epoch 132
Validation auc = 0.685446
Epoch 133
Validation auc = 0.692958
Epoch 134
Loss = 2.3990e-02, PNorm = 59.6697, GNorm = 0.4726, lr_0 = 8.0476e-04
Validation auc = 0.692019
Epoch 135
Validation auc = 0.702347
Epoch 136
Validation auc = 0.716901
Epoch 137
Validation auc = 0.704695
Epoch 138
Validation auc = 0.688263
Epoch 139
Loss = 5.8391e-02, PNorm = 59.7897, GNorm = 0.7678, lr_0 = 7.9788e-04
Validation auc = 0.680751
Epoch 140
Validation auc = 0.687793
Epoch 141
Validation auc = 0.685446
Epoch 142
Validation auc = 0.669953
Epoch 143
Validation auc = 0.669953
Epoch 144
Loss = 5.8172e-02, PNorm = 59.8976, GNorm = 1.4825, lr_0 = 7.9106e-04
Validation auc = 0.661033
Epoch 145
Validation auc = 0.653052
Epoch 146
Validation auc = 0.652113
Epoch 147
Validation auc = 0.648826
Epoch 148
Validation auc = 0.648826
Epoch 149
Loss = 2.5558e-02, PNorm = 60.0034, GNorm = 1.4313, lr_0 = 7.8430e-04
Validation auc = 0.654460
Epoch 150
Validation auc = 0.664319
Epoch 151
Validation auc = 0.674178
Epoch 152
Validation auc = 0.687793
Epoch 153
Validation auc = 0.690610
Epoch 154
Loss = 4.0682e-02, PNorm = 60.0785, GNorm = 1.3132, lr_0 = 7.7759e-04
Validation auc = 0.690141
Epoch 155
Validation auc = 0.671362
Epoch 156
Validation auc = 0.660563
Epoch 157
Validation auc = 0.654460
Epoch 158
Validation auc = 0.652582
Epoch 159
Loss = 3.3343e-02, PNorm = 60.1557, GNorm = 0.5283, lr_0 = 7.7095e-04
Validation auc = 0.650235
Epoch 160
Validation auc = 0.649296
Epoch 161
Validation auc = 0.650235
Epoch 162
Validation auc = 0.652582
Epoch 163
Validation auc = 0.657746
Epoch 164
Loss = 6.7958e-02, PNorm = 60.2339, GNorm = 0.4767, lr_0 = 7.6436e-04
Validation auc = 0.661972
Epoch 165
Validation auc = 0.664319
Epoch 166
Validation auc = 0.664789
Epoch 167
Validation auc = 0.661972
Epoch 168
Validation auc = 0.668075
Epoch 169
Loss = 2.3853e-02, PNorm = 60.2989, GNorm = 0.2240, lr_0 = 7.5782e-04
Validation auc = 0.669484
Epoch 170
Validation auc = 0.665728
Epoch 171
Validation auc = 0.661502
Epoch 172
Validation auc = 0.661033
Epoch 173
Validation auc = 0.676526
Epoch 174
Loss = 3.9889e-02, PNorm = 60.3763, GNorm = 1.9056, lr_0 = 7.5134e-04
Validation auc = 0.682160
Epoch 175
Validation auc = 0.681221
Epoch 176
Validation auc = 0.676056
Epoch 177
Validation auc = 0.672770
Epoch 178
Validation auc = 0.669484
Epoch 179
Loss = 5.9949e-02, PNorm = 60.4585, GNorm = 2.4471, lr_0 = 7.4492e-04
Validation auc = 0.668545
Epoch 180
Validation auc = 0.675117
Epoch 181
Validation auc = 0.685915
Epoch 182
Validation auc = 0.688732
Epoch 183
Validation auc = 0.686385
Epoch 184
Loss = 3.8207e-02, PNorm = 60.5550, GNorm = 0.5502, lr_0 = 7.3855e-04
Validation auc = 0.654460
Epoch 185
Validation auc = 0.629577
Epoch 186
Validation auc = 0.636620
Epoch 187
Validation auc = 0.634272
Epoch 188
Validation auc = 0.635211
Epoch 189
Loss = 2.4887e-02, PNorm = 60.6691, GNorm = 0.6147, lr_0 = 7.3224e-04
Validation auc = 0.634272
Epoch 190
Validation auc = 0.633333
Epoch 191
Validation auc = 0.631455
Epoch 192
Validation auc = 0.635211
Epoch 193
Validation auc = 0.639437
Epoch 194
Loss = 2.0968e-02, PNorm = 60.7578, GNorm = 0.9120, lr_0 = 7.2598e-04
Validation auc = 0.640845
Epoch 195
Validation auc = 0.657277
Epoch 196
Validation auc = 0.657746
Epoch 197
Validation auc = 0.652113
Epoch 198
Validation auc = 0.649296
Epoch 199
Loss = 9.4715e-02, PNorm = 60.8336, GNorm = 2.7746, lr_0 = 7.1977e-04
Validation auc = 0.650704
Epoch 200
Validation auc = 0.657746
Epoch 201
Validation auc = 0.659155
Epoch 202
Validation auc = 0.654930
Epoch 203
Validation auc = 0.655399
Epoch 204
Loss = 1.2845e-02, PNorm = 60.9235, GNorm = 1.1312, lr_0 = 7.1362e-04
Validation auc = 0.659624
Epoch 205
Validation auc = 0.686385
Epoch 206
Validation auc = 0.700469
Epoch 207
Validation auc = 0.700469
Epoch 208
Validation auc = 0.701878
Epoch 209
Loss = 1.7416e-02, PNorm = 61.0434, GNorm = 0.3927, lr_0 = 7.0752e-04
Validation auc = 0.700469
Epoch 210
Validation auc = 0.697183
Epoch 211
Validation auc = 0.700939
Epoch 212
Validation auc = 0.697183
Epoch 213
Validation auc = 0.683099
Epoch 214
Loss = 1.1571e-02, PNorm = 61.1706, GNorm = 0.2026, lr_0 = 7.0147e-04
Validation auc = 0.670892
Epoch 215
Validation auc = 0.656338
Epoch 216
Validation auc = 0.648826
Epoch 217
Validation auc = 0.643192
Epoch 218
Validation auc = 0.638967
Epoch 219
Loss = 1.1633e-02, PNorm = 61.2615, GNorm = 0.3617, lr_0 = 6.9548e-04
Validation auc = 0.637089
Epoch 220
Validation auc = 0.642254
Epoch 221
Validation auc = 0.646009
Epoch 222
Validation auc = 0.646948
Epoch 223
Validation auc = 0.650704
Epoch 224
Loss = 1.4432e-02, PNorm = 61.3333, GNorm = 0.6521, lr_0 = 6.8953e-04
Validation auc = 0.654930
Epoch 225
Validation auc = 0.655399
Epoch 226
Validation auc = 0.656338
Epoch 227
Validation auc = 0.660563
Epoch 228
Validation auc = 0.658685
Epoch 229
Loss = 5.8030e-03, PNorm = 61.3884, GNorm = 0.0901, lr_0 = 6.8364e-04
Validation auc = 0.654930
Epoch 230
Validation auc = 0.645070
Epoch 231
Validation auc = 0.639437
Epoch 232
Validation auc = 0.644601
Epoch 233
Validation auc = 0.658216
Epoch 234
Loss = 1.2353e-02, PNorm = 61.4341, GNorm = 0.3229, lr_0 = 6.7779e-04
Validation auc = 0.663380
Epoch 235
Validation auc = 0.659155
Epoch 236
Validation auc = 0.651174
Epoch 237
Validation auc = 0.646479
Epoch 238
Validation auc = 0.641315
Epoch 239
Loss = 4.5101e-03, PNorm = 61.4773, GNorm = 0.1896, lr_0 = 6.7200e-04
Validation auc = 0.638028
Epoch 240
Validation auc = 0.633333
Epoch 241
Validation auc = 0.632394
Epoch 242
Validation auc = 0.629108
Epoch 243
Validation auc = 0.630516
Epoch 244
Loss = 1.3552e-02, PNorm = 61.5139, GNorm = 0.0985, lr_0 = 6.6625e-04
Validation auc = 0.636150
Epoch 245
Validation auc = 0.638028
Epoch 246
Validation auc = 0.639906
Epoch 247
Validation auc = 0.639906
Epoch 248
Validation auc = 0.643662
Epoch 249
Loss = 4.4261e-03, PNorm = 61.5463, GNorm = 0.2227, lr_0 = 6.6056e-04
Validation auc = 0.644131
Epoch 250
Validation auc = 0.645540
Epoch 251
Validation auc = 0.647418
Epoch 252
Validation auc = 0.653521
Epoch 253
Validation auc = 0.653991
Epoch 254
Loss = 2.6295e-03, PNorm = 61.5745, GNorm = 0.1166, lr_0 = 6.5491e-04
Validation auc = 0.649765
Epoch 255
Validation auc = 0.640845
Epoch 256
Validation auc = 0.641315
Epoch 257
Validation auc = 0.649765
Epoch 258
Validation auc = 0.652113
Epoch 259
Loss = 8.5639e-03, PNorm = 61.6105, GNorm = 0.4937, lr_0 = 6.4931e-04
Validation auc = 0.660563
Epoch 260
Validation auc = 0.666667
Epoch 261
Validation auc = 0.667606
Epoch 262
Validation auc = 0.671831
Epoch 263
Validation auc = 0.674178
Epoch 264
Loss = 1.1255e-03, PNorm = 61.6511, GNorm = 0.0312, lr_0 = 6.4376e-04
Validation auc = 0.676995
Epoch 265
Validation auc = 0.678404
Epoch 266
Validation auc = 0.677934
Epoch 267
Validation auc = 0.676056
Epoch 268
Validation auc = 0.675587
Epoch 269
Loss = 1.8501e-02, PNorm = 61.6803, GNorm = 0.8394, lr_0 = 6.3826e-04
Validation auc = 0.677934
Epoch 270
Validation auc = 0.681690
Epoch 271
Validation auc = 0.686385
Epoch 272
Validation auc = 0.683099
Epoch 273
Validation auc = 0.677465
Epoch 274
Loss = 1.3778e-02, PNorm = 61.7191, GNorm = 0.5872, lr_0 = 6.3280e-04
Validation auc = 0.666197
Epoch 275
Validation auc = 0.660563
Epoch 276
Validation auc = 0.657746
Epoch 277
Validation auc = 0.656338
Epoch 278
Validation auc = 0.653991
Epoch 279
Loss = 3.4004e-03, PNorm = 61.7611, GNorm = 0.0933, lr_0 = 6.2739e-04
Validation auc = 0.652113
Epoch 280
Validation auc = 0.645540
Epoch 281
Validation auc = 0.641315
Epoch 282
Validation auc = 0.639906
Epoch 283
Validation auc = 0.637559
Epoch 284
Loss = 6.9355e-03, PNorm = 61.7919, GNorm = 0.1596, lr_0 = 6.2203e-04
Validation auc = 0.638967
Epoch 285
Validation auc = 0.642254
Epoch 286
Validation auc = 0.646009
Epoch 287
Validation auc = 0.649296
Epoch 288
Validation auc = 0.649296
Epoch 289
Loss = 4.1073e-03, PNorm = 61.8278, GNorm = 0.1446, lr_0 = 6.1671e-04
Validation auc = 0.651174
Epoch 290
Validation auc = 0.650235
Epoch 291
Validation auc = 0.649765
Epoch 292
Validation auc = 0.648357
Epoch 293
Validation auc = 0.648357
Epoch 294
Loss = 2.8663e-03, PNorm = 61.8542, GNorm = 0.0155, lr_0 = 6.1144e-04
Validation auc = 0.648826
Epoch 295
Validation auc = 0.652113
Epoch 296
Validation auc = 0.654930
Epoch 297
Validation auc = 0.654460
Epoch 298
Validation auc = 0.653052
Epoch 299
Loss = 1.4758e-03, PNorm = 61.8741, GNorm = 0.0456, lr_0 = 6.0621e-04
Validation auc = 0.649765
Model 0 best validation auc = 0.744131 on epoch 36
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.899458
Ensemble test auc = 0.899458
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.426761
Epoch 1
Validation auc = 0.486854
Epoch 2
Validation auc = 0.495305
Epoch 3
Validation auc = 0.524883
Epoch 4
Loss = 6.8324e-01, PNorm = 54.1568, GNorm = 1.5230, lr_0 = 6.5000e-04
Validation auc = 0.578873
Epoch 5
Validation auc = 0.612676
Epoch 6
Validation auc = 0.637559
Epoch 7
Validation auc = 0.669484
Epoch 8
Validation auc = 0.669484
Epoch 9
Loss = 6.5765e-01, PNorm = 54.3127, GNorm = 1.6325, lr_0 = 9.9743e-04
Validation auc = 0.700000
Epoch 10
Validation auc = 0.692019
Epoch 11
Validation auc = 0.702817
Epoch 12
Validation auc = 0.703756
Epoch 13
Validation auc = 0.695775
Epoch 14
Loss = 5.7556e-01, PNorm = 54.4978, GNorm = 1.2460, lr_0 = 9.8890e-04
Validation auc = 0.700000
Epoch 15
Validation auc = 0.694836
Epoch 16
Validation auc = 0.719249
Epoch 17
Validation auc = 0.700939
Epoch 18
Validation auc = 0.719249
Epoch 19
Loss = 4.6057e-01, PNorm = 54.6847, GNorm = 1.6773, lr_0 = 9.8045e-04
Validation auc = 0.719249
Epoch 20
Validation auc = 0.726761
Epoch 21
Validation auc = 0.730516
Epoch 22
Validation auc = 0.705634
Epoch 23
Validation auc = 0.710798
Epoch 24
Loss = 4.5212e-01, PNorm = 54.8838, GNorm = 2.3114, lr_0 = 9.7207e-04
Validation auc = 0.712207
Epoch 25
Validation auc = 0.706103
Epoch 26
Validation auc = 0.715962
Epoch 27
Validation auc = 0.730047
Epoch 28
Validation auc = 0.719249
Epoch 29
Loss = 4.3009e-01, PNorm = 55.0969, GNorm = 1.5697, lr_0 = 9.6376e-04
Validation auc = 0.722066
Epoch 30
Validation auc = 0.729108
Epoch 31
Validation auc = 0.726761
Epoch 32
Validation auc = 0.731925
Epoch 33
Validation auc = 0.738028
Epoch 34
Loss = 3.0302e-01, PNorm = 55.3116, GNorm = 0.6155, lr_0 = 9.5552e-04
Validation auc = 0.723474
Epoch 35
Validation auc = 0.721596
Epoch 36
Validation auc = 0.716432
Epoch 37
Validation auc = 0.722535
Epoch 38
Validation auc = 0.723005
Epoch 39
Loss = 3.2424e-01, PNorm = 55.5208, GNorm = 0.9749, lr_0 = 9.4735e-04
Validation auc = 0.712676
Epoch 40
Validation auc = 0.704225
Epoch 41
Validation auc = 0.704695
Epoch 42
Validation auc = 0.692958
Epoch 43
Validation auc = 0.686854
Epoch 44
Loss = 2.5178e-01, PNorm = 55.7655, GNorm = 1.3683, lr_0 = 9.3925e-04
Validation auc = 0.670423
Epoch 45
Validation auc = 0.702817
Epoch 46
Validation auc = 0.695305
Epoch 47
Validation auc = 0.686385
Epoch 48
Validation auc = 0.682160
Epoch 49
Loss = 2.0022e-01, PNorm = 56.0399, GNorm = 0.8771, lr_0 = 9.3122e-04
Validation auc = 0.684038
Epoch 50
Validation auc = 0.686385
Epoch 51
Validation auc = 0.680282
Epoch 52
Validation auc = 0.682160
Epoch 53
Validation auc = 0.680282
Epoch 54
Loss = 1.7601e-01, PNorm = 56.2880, GNorm = 2.2555, lr_0 = 9.2326e-04
Validation auc = 0.693427
Epoch 55
Validation auc = 0.727230
Epoch 56
Validation auc = 0.706103
Epoch 57
Validation auc = 0.686385
Epoch 58
Validation auc = 0.692488
Epoch 59
Loss = 2.1586e-01, PNorm = 56.5663, GNorm = 1.3405, lr_0 = 9.1537e-04
Validation auc = 0.693427
Epoch 60
Validation auc = 0.700939
Epoch 61
Validation auc = 0.704225
Epoch 62
Validation auc = 0.713146
Epoch 63
Validation auc = 0.707512
Epoch 64
Loss = 1.1354e-01, PNorm = 56.8240, GNorm = 0.6925, lr_0 = 9.0754e-04
Validation auc = 0.691080
Epoch 65
Validation auc = 0.725352
Epoch 66
Validation auc = 0.720657
Epoch 67
Validation auc = 0.726291
Epoch 68
Validation auc = 0.721596
Epoch 69
Loss = 1.6802e-01, PNorm = 57.0775, GNorm = 0.9157, lr_0 = 8.9978e-04
Validation auc = 0.715962
Epoch 70
Validation auc = 0.701408
Epoch 71
Validation auc = 0.713615
Epoch 72
Validation auc = 0.706103
Epoch 73
Validation auc = 0.682160
Epoch 74
Loss = 6.6268e-02, PNorm = 57.3016, GNorm = 1.1129, lr_0 = 8.9209e-04
Validation auc = 0.690141
Epoch 75
Validation auc = 0.687324
Epoch 76
Validation auc = 0.684977
Epoch 77
Validation auc = 0.700469
Epoch 78
Validation auc = 0.700939
Epoch 79
Loss = 7.8368e-02, PNorm = 57.4919, GNorm = 0.6721, lr_0 = 8.8447e-04
Validation auc = 0.691549
Epoch 80
Validation auc = 0.689202
Epoch 81
Validation auc = 0.710798
Epoch 82
Validation auc = 0.707042
Epoch 83
Validation auc = 0.715962
Epoch 84
Loss = 5.8297e-02, PNorm = 57.6811, GNorm = 1.2450, lr_0 = 8.7691e-04
Validation auc = 0.710798
Epoch 85
Validation auc = 0.705634
Epoch 86
Validation auc = 0.714085
Epoch 87
Validation auc = 0.695305
Epoch 88
Validation auc = 0.682629
Epoch 89
Loss = 9.2637e-02, PNorm = 57.8560, GNorm = 0.7376, lr_0 = 8.6941e-04
Validation auc = 0.690610
Epoch 90
Validation auc = 0.680282
Epoch 91
Validation auc = 0.680282
Epoch 92
Validation auc = 0.712676
Epoch 93
Validation auc = 0.692019
Epoch 94
Loss = 8.5558e-02, PNorm = 58.0221, GNorm = 1.3331, lr_0 = 8.6198e-04
Validation auc = 0.689671
Epoch 95
Validation auc = 0.685446
Epoch 96
Validation auc = 0.690141
Epoch 97
Validation auc = 0.692488
Epoch 98
Validation auc = 0.690610
Epoch 99
Loss = 5.7029e-02, PNorm = 58.2107, GNorm = 0.8693, lr_0 = 8.5461e-04
Validation auc = 0.698592
Epoch 100
Validation auc = 0.707981
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.683099
Epoch 103
Validation auc = 0.674178
Epoch 104
Loss = 1.6381e-01, PNorm = 58.3898, GNorm = 0.7164, lr_0 = 8.4730e-04
Validation auc = 0.685915
Epoch 105
Validation auc = 0.678404
Epoch 106
Validation auc = 0.665728
Epoch 107
Validation auc = 0.678873
Epoch 108
Validation auc = 0.676526
Epoch 109
Loss = 3.7629e-02, PNorm = 58.5316, GNorm = 0.5847, lr_0 = 8.4006e-04
Validation auc = 0.670423
Epoch 110
Validation auc = 0.666197
Epoch 111
Validation auc = 0.684977
Epoch 112
Validation auc = 0.687324
Epoch 113
Validation auc = 0.675587
Epoch 114
Loss = 7.5098e-02, PNorm = 58.6640, GNorm = 1.5613, lr_0 = 8.3288e-04
Validation auc = 0.675117
Epoch 115
Validation auc = 0.701878
Epoch 116
Validation auc = 0.687324
Epoch 117
Validation auc = 0.663380
Epoch 118
Validation auc = 0.645540
Epoch 119
Loss = 7.6196e-02, PNorm = 58.8028, GNorm = 0.3135, lr_0 = 8.2576e-04
Validation auc = 0.651174
Epoch 120
Validation auc = 0.654460
Epoch 121
Validation auc = 0.661972
Epoch 122
Validation auc = 0.663850
Epoch 123
Validation auc = 0.671362
Epoch 124
Loss = 5.0738e-02, PNorm = 58.9277, GNorm = 0.7355, lr_0 = 8.1870e-04
Validation auc = 0.672770
Epoch 125
Validation auc = 0.670892
Epoch 126
Validation auc = 0.656808
Epoch 127
Validation auc = 0.667606
Epoch 128
Validation auc = 0.683099
Epoch 129
Loss = 4.0370e-02, PNorm = 59.0401, GNorm = 1.6740, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.676995
Epoch 131
Validation auc = 0.677934
Epoch 132
Validation auc = 0.686854
Epoch 133
Validation auc = 0.691549
Epoch 134
Loss = 5.2968e-02, PNorm = 59.1583, GNorm = 1.0994, lr_0 = 8.0476e-04
Validation auc = 0.697653
Epoch 135
Validation auc = 0.697183
Epoch 136
Validation auc = 0.696714
Epoch 137
Validation auc = 0.689671
Epoch 138
Validation auc = 0.683568
Epoch 139
Loss = 1.8265e-02, PNorm = 59.2767, GNorm = 0.5598, lr_0 = 7.9788e-04
Validation auc = 0.677934
Epoch 140
Validation auc = 0.676526
Epoch 141
Validation auc = 0.672770
Epoch 142
Validation auc = 0.673239
Epoch 143
Validation auc = 0.667606
Epoch 144
Loss = 2.5293e-02, PNorm = 59.3651, GNorm = 0.6432, lr_0 = 7.9106e-04
Validation auc = 0.663850
Epoch 145
Validation auc = 0.668075
Epoch 146
Validation auc = 0.679343
Epoch 147
Validation auc = 0.692019
Epoch 148
Validation auc = 0.694836
Epoch 149
Loss = 5.2635e-02, PNorm = 59.4462, GNorm = 1.4164, lr_0 = 7.8430e-04
Validation auc = 0.684507
Epoch 150
Validation auc = 0.684977
Epoch 151
Validation auc = 0.681221
Epoch 152
Validation auc = 0.675587
Epoch 153
Validation auc = 0.680282
Epoch 154
Loss = 1.0116e-02, PNorm = 59.5116, GNorm = 0.4209, lr_0 = 7.7759e-04
Validation auc = 0.681690
Epoch 155
Validation auc = 0.687793
Epoch 156
Validation auc = 0.690610
Epoch 157
Validation auc = 0.690610
Epoch 158
Validation auc = 0.687324
Epoch 159
Loss = 2.4394e-02, PNorm = 59.5771, GNorm = 0.3186, lr_0 = 7.7095e-04
Validation auc = 0.682160
Epoch 160
Validation auc = 0.680751
Epoch 161
Validation auc = 0.670892
Epoch 162
Validation auc = 0.682160
Epoch 163
Validation auc = 0.699531
Epoch 164
Loss = 1.5561e-01, PNorm = 59.6474, GNorm = 5.5541, lr_0 = 7.6436e-04
Validation auc = 0.682160
Epoch 165
Validation auc = 0.689202
Epoch 166
Validation auc = 0.691080
Epoch 167
Validation auc = 0.669014
Epoch 168
Validation auc = 0.653052
Epoch 169
Loss = 4.4180e-02, PNorm = 59.8278, GNorm = 1.2347, lr_0 = 7.5782e-04
Validation auc = 0.654930
Epoch 170
Validation auc = 0.672770
Epoch 171
Validation auc = 0.690610
Epoch 172
Validation auc = 0.698122
Epoch 173
Validation auc = 0.702817
Epoch 174
Loss = 1.1400e-01, PNorm = 60.0427, GNorm = 1.6749, lr_0 = 7.5134e-04
Validation auc = 0.708920
Epoch 175
Validation auc = 0.701408
Epoch 176
Validation auc = 0.683568
Epoch 177
Validation auc = 0.673239
Epoch 178
Validation auc = 0.678404
Epoch 179
Loss = 2.7459e-02, PNorm = 60.1949, GNorm = 0.5492, lr_0 = 7.4492e-04
Validation auc = 0.673239
Epoch 180
Validation auc = 0.671831
Epoch 181
Validation auc = 0.670423
Epoch 182
Validation auc = 0.675587
Epoch 183
Validation auc = 0.681221
Epoch 184
Loss = 9.1549e-03, PNorm = 60.3127, GNorm = 0.1642, lr_0 = 7.3855e-04
Validation auc = 0.683099
Epoch 185
Validation auc = 0.686854
Epoch 186
Validation auc = 0.685446
Epoch 187
Validation auc = 0.684038
Epoch 188
Validation auc = 0.683568
Epoch 189
Loss = 8.6344e-03, PNorm = 60.3868, GNorm = 0.2042, lr_0 = 7.3224e-04
Validation auc = 0.675117
Epoch 190
Validation auc = 0.668545
Epoch 191
Validation auc = 0.672300
Epoch 192
Validation auc = 0.669484
Epoch 193
Validation auc = 0.670892
Epoch 194
Loss = 9.0319e-03, PNorm = 60.4460, GNorm = 0.3073, lr_0 = 7.2598e-04
Validation auc = 0.670423
Epoch 195
Validation auc = 0.671831
Epoch 196
Validation auc = 0.676526
Epoch 197
Validation auc = 0.674178
Epoch 198
Validation auc = 0.668545
Epoch 199
Loss = 1.0564e-02, PNorm = 60.5136, GNorm = 0.1392, lr_0 = 7.1977e-04
Validation auc = 0.665728
Epoch 200
Validation auc = 0.660094
Epoch 201
Validation auc = 0.668545
Epoch 202
Validation auc = 0.672770
Epoch 203
Validation auc = 0.678873
Epoch 204
Loss = 7.2652e-03, PNorm = 60.5814, GNorm = 0.1348, lr_0 = 7.1362e-04
Validation auc = 0.682629
Epoch 205
Validation auc = 0.684507
Epoch 206
Validation auc = 0.689671
Epoch 207
Validation auc = 0.690610
Epoch 208
Validation auc = 0.689202
Epoch 209
Loss = 5.4835e-02, PNorm = 60.6331, GNorm = 0.4556, lr_0 = 7.0752e-04
Validation auc = 0.692019
Epoch 210
Validation auc = 0.690141
Epoch 211
Validation auc = 0.687324
Epoch 212
Validation auc = 0.686385
Epoch 213
Validation auc = 0.685446
Epoch 214
Loss = 4.7956e-03, PNorm = 60.6804, GNorm = 0.1635, lr_0 = 7.0147e-04
Validation auc = 0.684038
Epoch 215
Validation auc = 0.684977
Epoch 216
Validation auc = 0.685446
Epoch 217
Validation auc = 0.682629
Epoch 218
Validation auc = 0.685915
Epoch 219
Loss = 5.1067e-03, PNorm = 60.7223, GNorm = 0.3329, lr_0 = 6.9548e-04
Validation auc = 0.691080
Epoch 220
Validation auc = 0.687793
Epoch 221
Validation auc = 0.690141
Epoch 222
Validation auc = 0.689671
Epoch 223
Validation auc = 0.693427
Epoch 224
Loss = 4.7385e-03, PNorm = 60.7603, GNorm = 0.0914, lr_0 = 6.8953e-04
Validation auc = 0.696714
Epoch 225
Validation auc = 0.700469
Epoch 226
Validation auc = 0.704225
Epoch 227
Validation auc = 0.709859
Epoch 228
Validation auc = 0.715493
Epoch 229
Loss = 1.9225e-02, PNorm = 60.8008, GNorm = 1.0954, lr_0 = 6.8364e-04
Validation auc = 0.719249
Epoch 230
Validation auc = 0.721127
Epoch 231
Validation auc = 0.707512
Epoch 232
Validation auc = 0.691549
Epoch 233
Validation auc = 0.678404
Epoch 234
Loss = 1.6653e-02, PNorm = 60.8568, GNorm = 0.1848, lr_0 = 6.7779e-04
Validation auc = 0.673239
Epoch 235
Validation auc = 0.671831
Epoch 236
Validation auc = 0.672300
Epoch 237
Validation auc = 0.674178
Epoch 238
Validation auc = 0.670423
Epoch 239
Loss = 2.7965e-03, PNorm = 60.9156, GNorm = 0.1585, lr_0 = 6.7200e-04
Validation auc = 0.672770
Epoch 240
Validation auc = 0.665258
Epoch 241
Validation auc = 0.666197
Epoch 242
Validation auc = 0.668075
Epoch 243
Validation auc = 0.669953
Epoch 244
Loss = 1.4463e-02, PNorm = 60.9610, GNorm = 0.5720, lr_0 = 6.6625e-04
Validation auc = 0.670423
Epoch 245
Validation auc = 0.673239
Epoch 246
Validation auc = 0.674178
Epoch 247
Validation auc = 0.676526
Epoch 248
Validation auc = 0.679343
Epoch 249
Loss = 3.2897e-03, PNorm = 60.9971, GNorm = 0.1045, lr_0 = 6.6056e-04
Validation auc = 0.684038
Epoch 250
Validation auc = 0.683568
Epoch 251
Validation auc = 0.675822
Epoch 252
Validation auc = 0.670892
Epoch 253
Validation auc = 0.666667
Epoch 254
Loss = 1.1342e-02, PNorm = 61.0323, GNorm = 0.9311, lr_0 = 6.5491e-04
Validation auc = 0.670892
Epoch 255
Validation auc = 0.676526
Epoch 256
Validation auc = 0.679812
Epoch 257
Validation auc = 0.683099
Epoch 258
Validation auc = 0.681221
Epoch 259
Loss = 1.1898e-03, PNorm = 61.0683, GNorm = 0.0320, lr_0 = 6.4931e-04
Validation auc = 0.680282
Epoch 260
Validation auc = 0.677465
Epoch 261
Validation auc = 0.674178
Epoch 262
Validation auc = 0.676056
Epoch 263
Validation auc = 0.677934
Epoch 264
Loss = 1.0786e-03, PNorm = 61.0966, GNorm = 0.0135, lr_0 = 6.4376e-04
Validation auc = 0.679812
Epoch 265
Validation auc = 0.680751
Epoch 266
Validation auc = 0.680282
Epoch 267
Validation auc = 0.679343
Epoch 268
Validation auc = 0.676526
Epoch 269
Loss = 6.1316e-04, PNorm = 61.1170, GNorm = 0.0180, lr_0 = 6.3826e-04
Validation auc = 0.676056
Epoch 270
Validation auc = 0.675587
Epoch 271
Validation auc = 0.674178
Epoch 272
Validation auc = 0.671831
Epoch 273
Validation auc = 0.670423
Epoch 274
Loss = 1.1892e-03, PNorm = 61.1287, GNorm = 0.1071, lr_0 = 6.3280e-04
Validation auc = 0.668545
Epoch 275
Validation auc = 0.670423
Epoch 276
Validation auc = 0.670423
Epoch 277
Validation auc = 0.670423
Epoch 278
Validation auc = 0.669953
Epoch 279
Loss = 2.7094e-04, PNorm = 61.1380, GNorm = 0.0120, lr_0 = 6.2739e-04
Validation auc = 0.669953
Epoch 280
Validation auc = 0.669953
Epoch 281
Validation auc = 0.669484
Epoch 282
Validation auc = 0.669484
Epoch 283
Validation auc = 0.667606
Epoch 284
Loss = 2.8600e-04, PNorm = 61.1442, GNorm = 0.0155, lr_0 = 6.2203e-04
Validation auc = 0.667136
Epoch 285
Validation auc = 0.666667
Epoch 286
Validation auc = 0.666667
Epoch 287
Validation auc = 0.666667
Epoch 288
Validation auc = 0.673709
Epoch 289
Loss = 3.2561e-04, PNorm = 61.1507, GNorm = 0.0129, lr_0 = 6.1671e-04
Validation auc = 0.675587
Epoch 290
Validation auc = 0.678873
Epoch 291
Validation auc = 0.681690
Epoch 292
Validation auc = 0.683099
Epoch 293
Validation auc = 0.684507
Epoch 294
Loss = 1.0212e-03, PNorm = 61.1598, GNorm = 0.0178, lr_0 = 6.1144e-04
Validation auc = 0.684507
Epoch 295
Validation auc = 0.682629
Epoch 296
Validation auc = 0.683568
Epoch 297
Validation auc = 0.685915
Epoch 298
Validation auc = 0.679343
Epoch 299
Loss = 3.1819e-03, PNorm = 61.1878, GNorm = 0.0839, lr_0 = 6.0621e-04
Validation auc = 0.672770
Model 0 best validation auc = 0.738028 on epoch 33
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.904877
Ensemble test auc = 0.904877
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.443192
Epoch 1
Validation auc = 0.465728
Epoch 2
Validation auc = 0.501878
Epoch 3
Validation auc = 0.555869
Epoch 4
Loss = 7.2379e-01, PNorm = 54.1542, GNorm = 1.6579, lr_0 = 6.5000e-04
Validation auc = 0.589671
Epoch 5
Validation auc = 0.616432
Epoch 6
Validation auc = 0.653991
Epoch 7
Validation auc = 0.662441
Epoch 8
Validation auc = 0.658685
Epoch 9
Loss = 5.3475e-01, PNorm = 54.3146, GNorm = 0.9070, lr_0 = 9.9743e-04
Validation auc = 0.663380
Epoch 10
Validation auc = 0.686385
Epoch 11
Validation auc = 0.686854
Epoch 12
Validation auc = 0.697653
Epoch 13
Validation auc = 0.670892
Epoch 14
Loss = 6.0139e-01, PNorm = 54.5079, GNorm = 1.3238, lr_0 = 9.8890e-04
Validation auc = 0.665258
Epoch 15
Validation auc = 0.674178
Epoch 16
Validation auc = 0.678404
Epoch 17
Validation auc = 0.689671
Epoch 18
Validation auc = 0.702347
Epoch 19
Loss = 5.2024e-01, PNorm = 54.7228, GNorm = 1.4444, lr_0 = 9.8045e-04
Validation auc = 0.709859
Epoch 20
Validation auc = 0.707512
Epoch 21
Validation auc = 0.719718
Epoch 22
Validation auc = 0.712676
Epoch 23
Validation auc = 0.722066
Epoch 24
Loss = 4.3700e-01, PNorm = 54.9213, GNorm = 1.7445, lr_0 = 9.7207e-04
Validation auc = 0.723005
Epoch 25
Validation auc = 0.711737
Epoch 26
Validation auc = 0.719249
Epoch 27
Validation auc = 0.722066
Epoch 28
Validation auc = 0.720657
Epoch 29
Loss = 4.3091e-01, PNorm = 55.1225, GNorm = 0.7929, lr_0 = 9.6376e-04
Validation auc = 0.722535
Epoch 30
Validation auc = 0.723005
Epoch 31
Validation auc = 0.727700
Epoch 32
Validation auc = 0.735681
Epoch 33
Validation auc = 0.725822
Epoch 34
Loss = 3.9887e-01, PNorm = 55.3334, GNorm = 1.1548, lr_0 = 9.5552e-04
Validation auc = 0.738967
Epoch 35
Validation auc = 0.718310
Epoch 36
Validation auc = 0.714554
Epoch 37
Validation auc = 0.716901
Epoch 38
Validation auc = 0.720657
Epoch 39
Loss = 3.4833e-01, PNorm = 55.5365, GNorm = 0.9556, lr_0 = 9.4735e-04
Validation auc = 0.722066
Epoch 40
Validation auc = 0.717371
Epoch 41
Validation auc = 0.702817
Epoch 42
Validation auc = 0.713146
Epoch 43
Validation auc = 0.714554
Epoch 44
Loss = 2.7559e-01, PNorm = 55.7883, GNorm = 1.2166, lr_0 = 9.3925e-04
Validation auc = 0.705164
Epoch 45
Validation auc = 0.714554
Epoch 46
Validation auc = 0.719249
Epoch 47
Validation auc = 0.710329
Epoch 48
Validation auc = 0.696714
Epoch 49
Loss = 3.3685e-01, PNorm = 56.0774, GNorm = 0.9918, lr_0 = 9.3122e-04
Validation auc = 0.700939
Epoch 50
Validation auc = 0.696244
Epoch 51
Validation auc = 0.699061
Epoch 52
Validation auc = 0.681690
Epoch 53
Validation auc = 0.682629
Epoch 54
Loss = 2.0990e-01, PNorm = 56.3670, GNorm = 0.8327, lr_0 = 9.2326e-04
Validation auc = 0.667136
Epoch 55
Validation auc = 0.679812
Epoch 56
Validation auc = 0.673709
Epoch 57
Validation auc = 0.692019
Epoch 58
Validation auc = 0.680751
Epoch 59
Loss = 2.8837e-01, PNorm = 56.6473, GNorm = 2.0520, lr_0 = 9.1537e-04
Validation auc = 0.697183
Epoch 60
Validation auc = 0.675587
Epoch 61
Validation auc = 0.705164
Epoch 62
Validation auc = 0.721127
Epoch 63
Validation auc = 0.698122
Epoch 64
Loss = 1.9068e-01, PNorm = 56.9379, GNorm = 1.2253, lr_0 = 9.0754e-04
Validation auc = 0.693427
Epoch 65
Validation auc = 0.694366
Epoch 66
Validation auc = 0.692958
Epoch 67
Validation auc = 0.684507
Epoch 68
Validation auc = 0.670423
Epoch 69
Loss = 9.3577e-02, PNorm = 57.2080, GNorm = 0.6036, lr_0 = 8.9978e-04
Validation auc = 0.671831
Epoch 70
Validation auc = 0.688263
Epoch 71
Validation auc = 0.699531
Epoch 72
Validation auc = 0.689671
Epoch 73
Validation auc = 0.692958
Epoch 74
Loss = 1.5725e-01, PNorm = 57.4504, GNorm = 1.0851, lr_0 = 8.9209e-04
Validation auc = 0.712676
Epoch 75
Validation auc = 0.698592
Epoch 76
Validation auc = 0.692019
Epoch 77
Validation auc = 0.695305
Epoch 78
Validation auc = 0.675117
Epoch 79
Loss = 9.7941e-02, PNorm = 57.6595, GNorm = 0.6780, lr_0 = 8.8447e-04
Validation auc = 0.663380
Epoch 80
Validation auc = 0.686385
Epoch 81
Validation auc = 0.655399
Epoch 82
Validation auc = 0.673239
Epoch 83
Validation auc = 0.689671
Epoch 84
Loss = 9.4556e-02, PNorm = 57.8758, GNorm = 0.6798, lr_0 = 8.7691e-04
Validation auc = 0.664319
Epoch 85
Validation auc = 0.661033
Epoch 86
Validation auc = 0.660563
Epoch 87
Validation auc = 0.661972
Epoch 88
Validation auc = 0.686385
Epoch 89
Loss = 5.8932e-02, PNorm = 58.0771, GNorm = 2.0806, lr_0 = 8.6941e-04
Validation auc = 0.682160
Epoch 90
Validation auc = 0.666667
Epoch 91
Validation auc = 0.676995
Epoch 92
Validation auc = 0.678404
Epoch 93
Validation auc = 0.667606
Epoch 94
Loss = 6.7494e-02, PNorm = 58.2564, GNorm = 1.2468, lr_0 = 8.6198e-04
Validation auc = 0.676056
Epoch 95
Validation auc = 0.674648
Epoch 96
Validation auc = 0.661502
Epoch 97
Validation auc = 0.697183
Epoch 98
Validation auc = 0.707981
Epoch 99
Loss = 4.0593e-02, PNorm = 58.4178, GNorm = 0.1650, lr_0 = 8.5461e-04
Validation auc = 0.683099
Epoch 100
Validation auc = 0.671831
Epoch 101
Validation auc = 0.684507
Epoch 102
Validation auc = 0.676056
Epoch 103
Validation auc = 0.684038
Epoch 104
Loss = 6.9135e-02, PNorm = 58.5702, GNorm = 1.4722, lr_0 = 8.4730e-04
Validation auc = 0.683099
Epoch 105
Validation auc = 0.662441
Epoch 106
Validation auc = 0.671362
Epoch 107
Validation auc = 0.660563
Epoch 108
Validation auc = 0.678404
Epoch 109
Loss = 5.0665e-02, PNorm = 58.7091, GNorm = 0.7918, lr_0 = 8.4006e-04
Validation auc = 0.686385
Epoch 110
Validation auc = 0.671362
Epoch 111
Validation auc = 0.667136
Epoch 112
Validation auc = 0.679343
Epoch 113
Validation auc = 0.674178
Epoch 114
Loss = 1.1826e-01, PNorm = 58.8694, GNorm = 1.4452, lr_0 = 8.3288e-04
Validation auc = 0.679343
Epoch 115
Validation auc = 0.717371
Epoch 116
Validation auc = 0.689671
Epoch 117
Validation auc = 0.663380
Epoch 118
Validation auc = 0.654930
Epoch 119
Loss = 1.3778e-01, PNorm = 58.9970, GNorm = 1.1093, lr_0 = 8.2576e-04
Validation auc = 0.663850
Epoch 120
Validation auc = 0.646948
Epoch 121
Validation auc = 0.645540
Epoch 122
Validation auc = 0.650235
Epoch 123
Validation auc = 0.680282
Epoch 124
Loss = 5.5217e-02, PNorm = 59.1492, GNorm = 1.0611, lr_0 = 8.1870e-04
Validation auc = 0.694366
Epoch 125
Validation auc = 0.678873
Epoch 126
Validation auc = 0.689202
Epoch 127
Validation auc = 0.694836
Epoch 128
Validation auc = 0.693897
Epoch 129
Loss = 4.9904e-02, PNorm = 59.2702, GNorm = 0.7441, lr_0 = 8.1170e-04
Validation auc = 0.684507
Epoch 130
Validation auc = 0.688732
Epoch 131
Validation auc = 0.702347
Epoch 132
Validation auc = 0.668075
Epoch 133
Validation auc = 0.668075
Epoch 134
Loss = 8.1880e-02, PNorm = 59.3833, GNorm = 0.5370, lr_0 = 8.0476e-04
Validation auc = 0.691549
Epoch 135
Validation auc = 0.695775
Epoch 136
Validation auc = 0.679343
Epoch 137
Validation auc = 0.677465
Epoch 138
Validation auc = 0.682629
Epoch 139
Loss = 3.2362e-02, PNorm = 59.5059, GNorm = 0.4224, lr_0 = 7.9788e-04
Validation auc = 0.690610
Epoch 140
Validation auc = 0.686854
Epoch 141
Validation auc = 0.684507
Epoch 142
Validation auc = 0.673709
Epoch 143
Validation auc = 0.673709
Epoch 144
Loss = 1.1654e-02, PNorm = 59.6160, GNorm = 0.2296, lr_0 = 7.9106e-04
Validation auc = 0.672770
Epoch 145
Validation auc = 0.677465
Epoch 146
Validation auc = 0.670892
Epoch 147
Validation auc = 0.664319
Epoch 148
Validation auc = 0.665728
Epoch 149
Loss = 2.0291e-02, PNorm = 59.7057, GNorm = 0.2359, lr_0 = 7.8430e-04
Validation auc = 0.681221
Epoch 150
Validation auc = 0.676995
Epoch 151
Validation auc = 0.679812
Epoch 152
Validation auc = 0.692019
Epoch 153
Validation auc = 0.704225
Epoch 154
Loss = 2.9368e-02, PNorm = 59.7967, GNorm = 0.4918, lr_0 = 7.7759e-04
Validation auc = 0.697183
Epoch 155
Validation auc = 0.695305
Epoch 156
Validation auc = 0.695775
Epoch 157
Validation auc = 0.695305
Epoch 158
Validation auc = 0.692958
Epoch 159
Loss = 2.4797e-02, PNorm = 59.8795, GNorm = 0.0648, lr_0 = 7.7095e-04
Validation auc = 0.681221
Epoch 160
Validation auc = 0.670423
Epoch 161
Validation auc = 0.667606
Epoch 162
Validation auc = 0.669014
Epoch 163
Validation auc = 0.666197
Epoch 164
Loss = 6.9750e-02, PNorm = 59.9612, GNorm = 1.6112, lr_0 = 7.6436e-04
Validation auc = 0.663380
Epoch 165
Validation auc = 0.665258
Epoch 166
Validation auc = 0.674648
Epoch 167
Validation auc = 0.675587
Epoch 168
Validation auc = 0.674648
Epoch 169
Loss = 2.7201e-02, PNorm = 60.0501, GNorm = 0.8862, lr_0 = 7.5782e-04
Validation auc = 0.678404
Epoch 170
Validation auc = 0.684977
Epoch 171
Validation auc = 0.688732
Epoch 172
Validation auc = 0.685915
Epoch 173
Validation auc = 0.671831
Epoch 174
Loss = 4.5436e-03, PNorm = 60.1329, GNorm = 0.1083, lr_0 = 7.5134e-04
Validation auc = 0.665728
Epoch 175
Validation auc = 0.663850
Epoch 176
Validation auc = 0.661033
Epoch 177
Validation auc = 0.677465
Epoch 178
Validation auc = 0.676995
Epoch 179
Loss = 7.4388e-03, PNorm = 60.2132, GNorm = 0.3240, lr_0 = 7.4492e-04
Validation auc = 0.675587
Epoch 180
Validation auc = 0.680282
Epoch 181
Validation auc = 0.689671
Epoch 182
Validation auc = 0.693897
Epoch 183
Validation auc = 0.684038
Epoch 184
Loss = 1.8159e-02, PNorm = 60.3007, GNorm = 0.5568, lr_0 = 7.3855e-04
Validation auc = 0.667136
Epoch 185
Validation auc = 0.656338
Epoch 186
Validation auc = 0.652582
Epoch 187
Validation auc = 0.659155
Epoch 188
Validation auc = 0.656808
Epoch 189
Loss = 5.9561e-03, PNorm = 60.3945, GNorm = 0.0871, lr_0 = 7.3224e-04
Validation auc = 0.651174
Epoch 190
Validation auc = 0.669014
Epoch 191
Validation auc = 0.681221
Epoch 192
Validation auc = 0.665728
Epoch 193
Validation auc = 0.646479
Epoch 194
Loss = 1.1268e-01, PNorm = 60.4830, GNorm = 0.1881, lr_0 = 7.2598e-04
Validation auc = 0.656808
Epoch 195
Validation auc = 0.665258
Epoch 196
Validation auc = 0.674178
Epoch 197
Validation auc = 0.672770
Epoch 198
Validation auc = 0.673239
Epoch 199
Loss = 4.1598e-02, PNorm = 60.6008, GNorm = 0.2806, lr_0 = 7.1977e-04
Validation auc = 0.676995
Epoch 200
Validation auc = 0.679343
Epoch 201
Validation auc = 0.673709
Epoch 202
Validation auc = 0.670423
Epoch 203
Validation auc = 0.669484
Epoch 204
Loss = 3.2875e-03, PNorm = 60.6977, GNorm = 0.0968, lr_0 = 7.1362e-04
Validation auc = 0.662441
Epoch 205
Validation auc = 0.669014
Epoch 206
Validation auc = 0.675117
Epoch 207
Validation auc = 0.684977
Epoch 208
Validation auc = 0.684977
Epoch 209
Loss = 1.3130e-02, PNorm = 60.7806, GNorm = 0.0421, lr_0 = 7.0752e-04
Validation auc = 0.674648
Epoch 210
Validation auc = 0.671831
Epoch 211
Validation auc = 0.669953
Epoch 212
Validation auc = 0.672300
Epoch 213
Validation auc = 0.683099
Epoch 214
Loss = 4.2356e-02, PNorm = 60.8659, GNorm = 0.8313, lr_0 = 7.0147e-04
Validation auc = 0.689202
Epoch 215
Validation auc = 0.684977
Epoch 216
Validation auc = 0.682629
Epoch 217
Validation auc = 0.678873
Epoch 218
Validation auc = 0.677934
Epoch 219
Loss = 1.0450e-02, PNorm = 60.9414, GNorm = 0.6378, lr_0 = 6.9548e-04
Validation auc = 0.686385
Epoch 220
Validation auc = 0.701408
Epoch 221
Validation auc = 0.710798
Epoch 222
Validation auc = 0.706103
Epoch 223
Validation auc = 0.702347
Epoch 224
Loss = 5.6326e-03, PNorm = 61.0015, GNorm = 0.3487, lr_0 = 6.8953e-04
Validation auc = 0.701878
Epoch 225
Validation auc = 0.702347
Epoch 226
Validation auc = 0.695305
Epoch 227
Validation auc = 0.674648
Epoch 228
Validation auc = 0.666197
Epoch 229
Loss = 1.0253e-02, PNorm = 61.0604, GNorm = 1.1067, lr_0 = 6.8364e-04
Validation auc = 0.666197
Epoch 230
Validation auc = 0.675117
Epoch 231
Validation auc = 0.678404
Epoch 232
Validation auc = 0.683568
Epoch 233
Validation auc = 0.685446
Epoch 234
Loss = 4.5094e-03, PNorm = 61.1087, GNorm = 0.1032, lr_0 = 6.7779e-04
Validation auc = 0.687324
Epoch 235
Validation auc = 0.686385
Epoch 236
Validation auc = 0.685915
Epoch 237
Validation auc = 0.683568
Epoch 238
Validation auc = 0.688263
Epoch 239
Loss = 2.0348e-03, PNorm = 61.1456, GNorm = 0.0202, lr_0 = 6.7200e-04
Validation auc = 0.697183
Epoch 240
Validation auc = 0.705164
Epoch 241
Validation auc = 0.703756
Epoch 242
Validation auc = 0.698592
Epoch 243
Validation auc = 0.692019
Epoch 244
Loss = 6.2208e-03, PNorm = 61.1855, GNorm = 0.0404, lr_0 = 6.6625e-04
Validation auc = 0.688732
Epoch 245
Validation auc = 0.688263
Epoch 246
Validation auc = 0.693427
Epoch 247
Validation auc = 0.701408
Epoch 248
Validation auc = 0.702817
Epoch 249
Loss = 4.8403e-03, PNorm = 61.2170, GNorm = 0.2161, lr_0 = 6.6056e-04
Validation auc = 0.702347
Epoch 250
Validation auc = 0.700939
Epoch 251
Validation auc = 0.695775
Epoch 252
Validation auc = 0.695775
Epoch 253
Validation auc = 0.694836
Epoch 254
Loss = 5.5217e-04, PNorm = 61.2374, GNorm = 0.0207, lr_0 = 6.5491e-04
Validation auc = 0.695305
Epoch 255
Validation auc = 0.697183
Epoch 256
Validation auc = 0.705634
Epoch 257
Validation auc = 0.709859
Epoch 258
Validation auc = 0.709859
Epoch 259
Loss = 7.7638e-03, PNorm = 61.2616, GNorm = 0.6592, lr_0 = 6.4931e-04
Validation auc = 0.704695
Epoch 260
Validation auc = 0.695775
Epoch 261
Validation auc = 0.684507
Epoch 262
Validation auc = 0.676056
Epoch 263
Validation auc = 0.669718
Epoch 264
Loss = 2.1493e-02, PNorm = 61.2892, GNorm = 0.1347, lr_0 = 6.4376e-04
Validation auc = 0.692958
Epoch 265
Validation auc = 0.705164
Epoch 266
Validation auc = 0.708920
Epoch 267
Validation auc = 0.700939
Epoch 268
Validation auc = 0.690141
Epoch 269
Loss = 2.3736e-03, PNorm = 61.3416, GNorm = 0.2364, lr_0 = 6.3826e-04
Validation auc = 0.687793
Epoch 270
Validation auc = 0.681690
Epoch 271
Validation auc = 0.671831
Epoch 272
Validation auc = 0.669484
Epoch 273
Validation auc = 0.661972
Epoch 274
Loss = 5.2646e-03, PNorm = 61.4197, GNorm = 0.1990, lr_0 = 6.3280e-04
Validation auc = 0.659624
Epoch 275
Validation auc = 0.657746
Epoch 276
Validation auc = 0.659155
Epoch 277
Validation auc = 0.661033
Epoch 278
Validation auc = 0.662441
Epoch 279
Loss = 1.9675e-03, PNorm = 61.4967, GNorm = 0.0382, lr_0 = 6.2739e-04
Validation auc = 0.665258
Epoch 280
Validation auc = 0.666667
Epoch 281
Validation auc = 0.666197
Epoch 282
Validation auc = 0.667606
Epoch 283
Validation auc = 0.665728
Epoch 284
Loss = 2.0673e-03, PNorm = 61.5386, GNorm = 0.1307, lr_0 = 6.2203e-04
Validation auc = 0.662441
Epoch 285
Validation auc = 0.659624
Epoch 286
Validation auc = 0.656808
Epoch 287
Validation auc = 0.662207
Epoch 288
Validation auc = 0.666667
Epoch 289
Loss = 4.3608e-03, PNorm = 61.5691, GNorm = 0.3313, lr_0 = 6.1671e-04
Validation auc = 0.668075
Epoch 290
Validation auc = 0.666197
Epoch 291
Validation auc = 0.663380
Epoch 292
Validation auc = 0.662441
Epoch 293
Validation auc = 0.661033
Epoch 294
Loss = 8.7220e-04, PNorm = 61.5868, GNorm = 0.0334, lr_0 = 6.1144e-04
Validation auc = 0.659624
Epoch 295
Validation auc = 0.657277
Epoch 296
Validation auc = 0.657746
Epoch 297
Validation auc = 0.658685
Epoch 298
Validation auc = 0.661502
Epoch 299
Loss = 5.3270e-04, PNorm = 61.6033, GNorm = 0.0142, lr_0 = 6.0621e-04
Validation auc = 0.664789
Model 0 best validation auc = 0.738967 on epoch 34
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.901264
Ensemble test auc = 0.901264
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.450235
Epoch 1
Validation auc = 0.460094
Epoch 2
Validation auc = 0.509859
Epoch 3
Validation auc = 0.547887
Epoch 4
Loss = 6.5607e-01, PNorm = 54.1589, GNorm = 1.7428, lr_0 = 6.5000e-04
Validation auc = 0.587324
Epoch 5
Validation auc = 0.633803
Epoch 6
Validation auc = 0.637559
Epoch 7
Validation auc = 0.660094
Epoch 8
Validation auc = 0.663380
Epoch 9
Loss = 5.7331e-01, PNorm = 54.3214, GNorm = 2.4185, lr_0 = 9.9743e-04
Validation auc = 0.695305
Epoch 10
Validation auc = 0.691549
Epoch 11
Validation auc = 0.690141
Epoch 12
Validation auc = 0.716901
Epoch 13
Validation auc = 0.706573
Epoch 14
Loss = 4.8985e-01, PNorm = 54.5234, GNorm = 1.8502, lr_0 = 9.8890e-04
Validation auc = 0.712207
Epoch 15
Validation auc = 0.719249
Epoch 16
Validation auc = 0.712207
Epoch 17
Validation auc = 0.719249
Epoch 18
Validation auc = 0.727700
Epoch 19
Loss = 4.9403e-01, PNorm = 54.7272, GNorm = 1.3934, lr_0 = 9.8045e-04
Validation auc = 0.728169
Epoch 20
Validation auc = 0.725822
Epoch 21
Validation auc = 0.699531
Epoch 22
Validation auc = 0.719718
Epoch 23
Validation auc = 0.694836
Epoch 24
Loss = 5.4983e-01, PNorm = 54.9117, GNorm = 1.7688, lr_0 = 9.7207e-04
Validation auc = 0.717371
Epoch 25
Validation auc = 0.730986
Epoch 26
Validation auc = 0.721596
Epoch 27
Validation auc = 0.730047
Epoch 28
Validation auc = 0.729577
Epoch 29
Loss = 3.9301e-01, PNorm = 55.1213, GNorm = 0.7615, lr_0 = 9.6376e-04
Validation auc = 0.734742
Epoch 30
Validation auc = 0.732394
Epoch 31
Validation auc = 0.732864
Epoch 32
Validation auc = 0.731455
Epoch 33
Validation auc = 0.728169
Epoch 34
Loss = 3.1570e-01, PNorm = 55.3012, GNorm = 0.7762, lr_0 = 9.5552e-04
Validation auc = 0.730986
Epoch 35
Validation auc = 0.728169
Epoch 36
Validation auc = 0.730047
Epoch 37
Validation auc = 0.725352
Epoch 38
Validation auc = 0.721596
Epoch 39
Loss = 3.9060e-01, PNorm = 55.4585, GNorm = 1.6018, lr_0 = 9.4735e-04
Validation auc = 0.713146
Epoch 40
Validation auc = 0.710329
Epoch 41
Validation auc = 0.718779
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.694366
Epoch 44
Loss = 3.7383e-01, PNorm = 55.6675, GNorm = 1.4754, lr_0 = 9.3925e-04
Validation auc = 0.677465
Epoch 45
Validation auc = 0.672770
Epoch 46
Validation auc = 0.682629
Epoch 47
Validation auc = 0.691549
Epoch 48
Validation auc = 0.702347
Epoch 49
Loss = 2.3278e-01, PNorm = 55.8985, GNorm = 1.4488, lr_0 = 9.3122e-04
Validation auc = 0.702817
Epoch 50
Validation auc = 0.706573
Epoch 51
Validation auc = 0.706573
Epoch 52
Validation auc = 0.704225
Epoch 53
Validation auc = 0.701878
Epoch 54
Loss = 2.7351e-01, PNorm = 56.1432, GNorm = 0.7504, lr_0 = 9.2326e-04
Validation auc = 0.698122
Epoch 55
Validation auc = 0.690141
Epoch 56
Validation auc = 0.684038
Epoch 57
Validation auc = 0.679343
Epoch 58
Validation auc = 0.694836
Epoch 59
Loss = 1.8503e-01, PNorm = 56.3990, GNorm = 0.7384, lr_0 = 9.1537e-04
Validation auc = 0.682160
Epoch 60
Validation auc = 0.704225
Epoch 61
Validation auc = 0.717840
Epoch 62
Validation auc = 0.717840
Epoch 63
Validation auc = 0.718779
Epoch 64
Loss = 1.8271e-01, PNorm = 56.6330, GNorm = 0.5910, lr_0 = 9.0754e-04
Validation auc = 0.726761
Epoch 65
Validation auc = 0.727230
Epoch 66
Validation auc = 0.715962
Epoch 67
Validation auc = 0.723944
Epoch 68
Validation auc = 0.732394
Epoch 69
Loss = 1.4918e-01, PNorm = 56.8710, GNorm = 0.6943, lr_0 = 8.9978e-04
Validation auc = 0.704695
Epoch 70
Validation auc = 0.698592
Epoch 71
Validation auc = 0.702347
Epoch 72
Validation auc = 0.710329
Epoch 73
Validation auc = 0.701878
Epoch 74
Loss = 1.4653e-01, PNorm = 57.1185, GNorm = 1.2638, lr_0 = 8.9209e-04
Validation auc = 0.707512
Epoch 75
Validation auc = 0.704695
Epoch 76
Validation auc = 0.715023
Epoch 77
Validation auc = 0.705634
Epoch 78
Validation auc = 0.695305
Epoch 79
Loss = 6.7531e-02, PNorm = 57.3257, GNorm = 1.0796, lr_0 = 8.8447e-04
Validation auc = 0.698592
Epoch 80
Validation auc = 0.703286
Epoch 81
Validation auc = 0.688732
Epoch 82
Validation auc = 0.690141
Epoch 83
Validation auc = 0.705634
Epoch 84
Loss = 2.0041e-01, PNorm = 57.5056, GNorm = 3.4134, lr_0 = 8.7691e-04
Validation auc = 0.674178
Epoch 85
Validation auc = 0.704695
Epoch 86
Validation auc = 0.718779
Epoch 87
Validation auc = 0.715962
Epoch 88
Validation auc = 0.719249
Epoch 89
Loss = 1.5114e-01, PNorm = 57.7127, GNorm = 1.1343, lr_0 = 8.6941e-04
Validation auc = 0.731455
Epoch 90
Validation auc = 0.736150
Epoch 91
Validation auc = 0.724413
Epoch 92
Validation auc = 0.712207
Epoch 93
Validation auc = 0.709859
Epoch 94
Loss = 1.5213e-01, PNorm = 57.8969, GNorm = 0.7167, lr_0 = 8.6198e-04
Validation auc = 0.716432
Epoch 95
Validation auc = 0.705634
Epoch 96
Validation auc = 0.692019
Epoch 97
Validation auc = 0.681690
Epoch 98
Validation auc = 0.678873
Epoch 99
Loss = 5.7093e-02, PNorm = 58.0516, GNorm = 0.6405, lr_0 = 8.5461e-04
Validation auc = 0.684507
Epoch 100
Validation auc = 0.697653
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.719249
Epoch 103
Validation auc = 0.726761
Epoch 104
Loss = 6.9280e-02, PNorm = 58.1970, GNorm = 1.3849, lr_0 = 8.4730e-04
Validation auc = 0.729108
Epoch 105
Validation auc = 0.730516
Epoch 106
Validation auc = 0.715493
Epoch 107
Validation auc = 0.714085
Epoch 108
Validation auc = 0.698592
Epoch 109
Loss = 3.6486e-02, PNorm = 58.3309, GNorm = 0.8478, lr_0 = 8.4006e-04
Validation auc = 0.691549
Epoch 110
Validation auc = 0.714085
Epoch 111
Validation auc = 0.714554
Epoch 112
Validation auc = 0.717840
Epoch 113
Validation auc = 0.717371
Epoch 114
Loss = 5.4264e-02, PNorm = 58.4721, GNorm = 1.4185, lr_0 = 8.3288e-04
Validation auc = 0.711737
Epoch 115
Validation auc = 0.699061
Epoch 116
Validation auc = 0.691080
Epoch 117
Validation auc = 0.701408
Epoch 118
Validation auc = 0.713146
Epoch 119
Loss = 7.8331e-02, PNorm = 58.6153, GNorm = 1.9264, lr_0 = 8.2576e-04
Validation auc = 0.710329
Epoch 120
Validation auc = 0.699531
Epoch 121
Validation auc = 0.699531
Epoch 122
Validation auc = 0.706573
Epoch 123
Validation auc = 0.703286
Epoch 124
Loss = 5.0704e-02, PNorm = 58.7447, GNorm = 0.4347, lr_0 = 8.1870e-04
Validation auc = 0.698122
Epoch 125
Validation auc = 0.697183
Epoch 126
Validation auc = 0.699061
Epoch 127
Validation auc = 0.696714
Epoch 128
Validation auc = 0.691549
Epoch 129
Loss = 2.3987e-02, PNorm = 58.8567, GNorm = 0.6821, lr_0 = 8.1170e-04
Validation auc = 0.692019
Epoch 130
Validation auc = 0.700000
Epoch 131
Validation auc = 0.701408
Epoch 132
Validation auc = 0.702347
Epoch 133
Validation auc = 0.736150
Epoch 134
Loss = 7.6559e-02, PNorm = 58.9775, GNorm = 2.6273, lr_0 = 8.0476e-04
Validation auc = 0.738967
Epoch 135
Validation auc = 0.728638
Epoch 136
Validation auc = 0.717840
Epoch 137
Validation auc = 0.714554
Epoch 138
Validation auc = 0.709859
Epoch 139
Loss = 4.1133e-02, PNorm = 59.1201, GNorm = 0.4945, lr_0 = 7.9788e-04
Validation auc = 0.703286
Epoch 140
Validation auc = 0.706573
Epoch 141
Validation auc = 0.707512
Epoch 142
Validation auc = 0.701878
Epoch 143
Validation auc = 0.692019
Epoch 144
Loss = 6.2187e-02, PNorm = 59.2502, GNorm = 0.5957, lr_0 = 7.9106e-04
Validation auc = 0.695775
Epoch 145
Validation auc = 0.693427
Epoch 146
Validation auc = 0.682160
Epoch 147
Validation auc = 0.678873
Epoch 148
Validation auc = 0.674178
Epoch 149
Loss = 4.4444e-02, PNorm = 59.3429, GNorm = 1.4602, lr_0 = 7.8430e-04
Validation auc = 0.688263
Epoch 150
Validation auc = 0.693427
Epoch 151
Validation auc = 0.691080
Epoch 152
Validation auc = 0.702347
Epoch 153
Validation auc = 0.707981
Epoch 154
Loss = 1.4468e-02, PNorm = 59.4366, GNorm = 0.3662, lr_0 = 7.7759e-04
Validation auc = 0.709859
Epoch 155
Validation auc = 0.706573
Epoch 156
Validation auc = 0.702817
Epoch 157
Validation auc = 0.712207
Epoch 158
Validation auc = 0.717371
Epoch 159
Loss = 2.4130e-02, PNorm = 59.5226, GNorm = 1.1220, lr_0 = 7.7095e-04
Validation auc = 0.709390
Epoch 160
Validation auc = 0.693427
Epoch 161
Validation auc = 0.686854
Epoch 162
Validation auc = 0.682160
Epoch 163
Validation auc = 0.684038
Epoch 164
Loss = 4.8951e-02, PNorm = 59.6162, GNorm = 1.9854, lr_0 = 7.6436e-04
Validation auc = 0.683568
Epoch 165
Validation auc = 0.668545
Epoch 166
Validation auc = 0.656338
Epoch 167
Validation auc = 0.662911
Epoch 168
Validation auc = 0.667136
Epoch 169
Loss = 8.6544e-03, PNorm = 59.7160, GNorm = 0.2539, lr_0 = 7.5782e-04
Validation auc = 0.661972
Epoch 170
Validation auc = 0.660094
Epoch 171
Validation auc = 0.655869
Epoch 172
Validation auc = 0.659155
Epoch 173
Validation auc = 0.661502
Epoch 174
Loss = 4.5113e-02, PNorm = 59.7914, GNorm = 0.1324, lr_0 = 7.5134e-04
Validation auc = 0.632864
Epoch 175
Validation auc = 0.625822
Epoch 176
Validation auc = 0.659624
Epoch 177
Validation auc = 0.676056
Epoch 178
Validation auc = 0.684507
Epoch 179
Loss = 1.9604e-02, PNorm = 59.9170, GNorm = 0.4389, lr_0 = 7.4492e-04
Validation auc = 0.699061
Epoch 180
Validation auc = 0.714554
Epoch 181
Validation auc = 0.722066
Epoch 182
Validation auc = 0.708920
Epoch 183
Validation auc = 0.690610
Epoch 184
Loss = 5.2616e-02, PNorm = 60.0530, GNorm = 0.7933, lr_0 = 7.3855e-04
Validation auc = 0.680751
Epoch 185
Validation auc = 0.669953
Epoch 186
Validation auc = 0.675587
Epoch 187
Validation auc = 0.691549
Epoch 188
Validation auc = 0.695305
Epoch 189
Loss = 4.6227e-02, PNorm = 60.1487, GNorm = 2.6310, lr_0 = 7.3224e-04
Validation auc = 0.691080
Epoch 190
Validation auc = 0.687793
Epoch 191
Validation auc = 0.701878
Epoch 192
Validation auc = 0.674648
Epoch 193
Validation auc = 0.665728
Epoch 194
Loss = 2.7810e-02, PNorm = 60.2467, GNorm = 1.1075, lr_0 = 7.2598e-04
Validation auc = 0.674648
Epoch 195
Validation auc = 0.676526
Epoch 196
Validation auc = 0.678873
Epoch 197
Validation auc = 0.668545
Epoch 198
Validation auc = 0.667606
Epoch 199
Loss = 3.0006e-02, PNorm = 60.3416, GNorm = 1.3251, lr_0 = 7.1977e-04
Validation auc = 0.681221
Epoch 200
Validation auc = 0.692019
Epoch 201
Validation auc = 0.691549
Epoch 202
Validation auc = 0.683568
Epoch 203
Validation auc = 0.676995
Epoch 204
Loss = 1.7430e-02, PNorm = 60.4217, GNorm = 0.4105, lr_0 = 7.1362e-04
Validation auc = 0.675587
Epoch 205
Validation auc = 0.672770
Epoch 206
Validation auc = 0.669014
Epoch 207
Validation auc = 0.662911
Epoch 208
Validation auc = 0.657277
Epoch 209
Loss = 1.8925e-02, PNorm = 60.4906, GNorm = 0.0751, lr_0 = 7.0752e-04
Validation auc = 0.656338
Epoch 210
Validation auc = 0.661502
Epoch 211
Validation auc = 0.662441
Epoch 212
Validation auc = 0.663380
Epoch 213
Validation auc = 0.665728
Epoch 214
Loss = 4.3499e-03, PNorm = 60.5464, GNorm = 0.1137, lr_0 = 7.0147e-04
Validation auc = 0.665728
Epoch 215
Validation auc = 0.666197
Epoch 216
Validation auc = 0.669484
Epoch 217
Validation auc = 0.676526
Epoch 218
Validation auc = 0.680282
Epoch 219
Loss = 1.5556e-03, PNorm = 60.5828, GNorm = 0.0181, lr_0 = 6.9548e-04
Validation auc = 0.681690
Epoch 220
Validation auc = 0.687324
Epoch 221
Validation auc = 0.692019
Epoch 222
Validation auc = 0.694366
Epoch 223
Validation auc = 0.694366
Epoch 224
Loss = 5.2682e-03, PNorm = 60.6133, GNorm = 0.2643, lr_0 = 6.8953e-04
Validation auc = 0.687324
Epoch 225
Validation auc = 0.687793
Epoch 226
Validation auc = 0.685915
Epoch 227
Validation auc = 0.685446
Epoch 228
Validation auc = 0.684507
Epoch 229
Loss = 6.4570e-03, PNorm = 60.6478, GNorm = 0.1769, lr_0 = 6.8364e-04
Validation auc = 0.683099
Epoch 230
Validation auc = 0.686854
Epoch 231
Validation auc = 0.694836
Epoch 232
Validation auc = 0.694836
Epoch 233
Validation auc = 0.681221
Epoch 234
Loss = 3.6579e-03, PNorm = 60.6848, GNorm = 0.1594, lr_0 = 6.7779e-04
Validation auc = 0.676995
Epoch 235
Validation auc = 0.673709
Epoch 236
Validation auc = 0.675117
Epoch 237
Validation auc = 0.676056
Epoch 238
Validation auc = 0.681221
Epoch 239
Loss = 6.1776e-03, PNorm = 60.7300, GNorm = 0.4834, lr_0 = 6.7200e-04
Validation auc = 0.678404
Epoch 240
Validation auc = 0.677465
Epoch 241
Validation auc = 0.676056
Epoch 242
Validation auc = 0.675117
Epoch 243
Validation auc = 0.676056
Epoch 244
Loss = 3.9537e-03, PNorm = 60.7705, GNorm = 0.0921, lr_0 = 6.6625e-04
Validation auc = 0.676056
Epoch 245
Validation auc = 0.676056
Epoch 246
Validation auc = 0.670423
Epoch 247
Validation auc = 0.666197
Epoch 248
Validation auc = 0.660094
Epoch 249
Loss = 8.8563e-03, PNorm = 60.8188, GNorm = 0.8072, lr_0 = 6.6056e-04
Validation auc = 0.660094
Epoch 250
Validation auc = 0.680282
Epoch 251
Validation auc = 0.682629
Epoch 252
Validation auc = 0.674178
Epoch 253
Validation auc = 0.667606
Epoch 254
Loss = 1.8548e-03, PNorm = 60.9098, GNorm = 0.1687, lr_0 = 6.5491e-04
Validation auc = 0.658216
Epoch 255
Validation auc = 0.660563
Epoch 256
Validation auc = 0.663850
Epoch 257
Validation auc = 0.657277
Epoch 258
Validation auc = 0.661033
Epoch 259
Loss = 5.1635e-03, PNorm = 61.0093, GNorm = 0.1778, lr_0 = 6.4931e-04
Validation auc = 0.675117
Epoch 260
Validation auc = 0.684507
Epoch 261
Validation auc = 0.689671
Epoch 262
Validation auc = 0.693427
Epoch 263
Validation auc = 0.694836
Epoch 264
Loss = 4.0347e-03, PNorm = 61.0894, GNorm = 0.0702, lr_0 = 6.4376e-04
Validation auc = 0.698122
Epoch 265
Validation auc = 0.697653
Epoch 266
Validation auc = 0.694836
Epoch 267
Validation auc = 0.691549
Epoch 268
Validation auc = 0.690141
Epoch 269
Loss = 3.7977e-03, PNorm = 61.1503, GNorm = 0.0294, lr_0 = 6.3826e-04
Validation auc = 0.683099
Epoch 270
Validation auc = 0.682160
Epoch 271
Validation auc = 0.680751
Epoch 272
Validation auc = 0.685446
Epoch 273
Validation auc = 0.689671
Epoch 274
Loss = 2.4398e-03, PNorm = 61.1878, GNorm = 0.1521, lr_0 = 6.3280e-04
Validation auc = 0.693427
Epoch 275
Validation auc = 0.692958
Epoch 276
Validation auc = 0.690610
Epoch 277
Validation auc = 0.689202
Epoch 278
Validation auc = 0.687793
Epoch 279
Loss = 1.9367e-03, PNorm = 61.2112, GNorm = 0.1408, lr_0 = 6.2739e-04
Validation auc = 0.686854
Epoch 280
Validation auc = 0.684038
Epoch 281
Validation auc = 0.684977
Epoch 282
Validation auc = 0.684038
Epoch 283
Validation auc = 0.681690
Epoch 284
Loss = 1.4690e-02, PNorm = 61.2282, GNorm = 0.8419, lr_0 = 6.2203e-04
Validation auc = 0.684272
Epoch 285
Validation auc = 0.689671
Epoch 286
Validation auc = 0.692019
Epoch 287
Validation auc = 0.692958
Epoch 288
Validation auc = 0.693897
Epoch 289
Loss = 1.4705e-03, PNorm = 61.2539, GNorm = 0.0506, lr_0 = 6.1671e-04
Validation auc = 0.690141
Epoch 290
Validation auc = 0.690141
Epoch 291
Validation auc = 0.691549
Epoch 292
Validation auc = 0.692958
Epoch 293
Validation auc = 0.692958
Epoch 294
Loss = 8.6970e-04, PNorm = 61.2714, GNorm = 0.0337, lr_0 = 6.1144e-04
Validation auc = 0.692958
Epoch 295
Validation auc = 0.694836
Epoch 296
Validation auc = 0.696244
Epoch 297
Validation auc = 0.695305
Epoch 298
Validation auc = 0.695305
Epoch 299
Loss = 6.8478e-04, PNorm = 61.2842, GNorm = 0.0398, lr_0 = 6.0621e-04
Validation auc = 0.695305
Model 0 best validation auc = 0.738967 on epoch 134
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.854907
Ensemble test auc = 0.854907
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.436620
Epoch 1
Validation auc = 0.454460
Epoch 2
Validation auc = 0.508920
Epoch 3
Validation auc = 0.553052
Epoch 4
Loss = 7.0855e-01, PNorm = 54.1598, GNorm = 2.0248, lr_0 = 6.5000e-04
Validation auc = 0.625352
Epoch 5
Validation auc = 0.635681
Epoch 6
Validation auc = 0.664789
Epoch 7
Validation auc = 0.654930
Epoch 8
Validation auc = 0.664319
Epoch 9
Loss = 6.5989e-01, PNorm = 54.3157, GNorm = 4.5746, lr_0 = 9.9743e-04
Validation auc = 0.689202
Epoch 10
Validation auc = 0.697653
Epoch 11
Validation auc = 0.676526
Epoch 12
Validation auc = 0.672770
Epoch 13
Validation auc = 0.672770
Epoch 14
Loss = 6.1878e-01, PNorm = 54.4963, GNorm = 0.8499, lr_0 = 9.8890e-04
Validation auc = 0.679343
Epoch 15
Validation auc = 0.686385
Epoch 16
Validation auc = 0.692019
Epoch 17
Validation auc = 0.697183
Epoch 18
Validation auc = 0.696244
Epoch 19
Loss = 5.5207e-01, PNorm = 54.6782, GNorm = 0.7801, lr_0 = 9.8045e-04
Validation auc = 0.707512
Epoch 20
Validation auc = 0.715023
Epoch 21
Validation auc = 0.715023
Epoch 22
Validation auc = 0.707042
Epoch 23
Validation auc = 0.711268
Epoch 24
Loss = 4.3601e-01, PNorm = 54.8642, GNorm = 1.0626, lr_0 = 9.7207e-04
Validation auc = 0.710329
Epoch 25
Validation auc = 0.715493
Epoch 26
Validation auc = 0.727230
Epoch 27
Validation auc = 0.731455
Epoch 28
Validation auc = 0.727230
Epoch 29
Loss = 4.3894e-01, PNorm = 55.0480, GNorm = 0.6745, lr_0 = 9.6376e-04
Validation auc = 0.737559
Epoch 30
Validation auc = 0.732394
Epoch 31
Validation auc = 0.729108
Epoch 32
Validation auc = 0.727700
Epoch 33
Validation auc = 0.732864
Epoch 34
Loss = 4.4702e-01, PNorm = 55.2282, GNorm = 0.8562, lr_0 = 9.5552e-04
Validation auc = 0.724883
Epoch 35
Validation auc = 0.724413
Epoch 36
Validation auc = 0.718310
Epoch 37
Validation auc = 0.717371
Epoch 38
Validation auc = 0.714085
Epoch 39
Loss = 4.0166e-01, PNorm = 55.4280, GNorm = 0.8492, lr_0 = 9.4735e-04
Validation auc = 0.713146
Epoch 40
Validation auc = 0.721596
Epoch 41
Validation auc = 0.730047
Epoch 42
Validation auc = 0.715023
Epoch 43
Validation auc = 0.709859
Epoch 44
Loss = 2.7623e-01, PNorm = 55.6465, GNorm = 0.6943, lr_0 = 9.3925e-04
Validation auc = 0.696714
Epoch 45
Validation auc = 0.696714
Epoch 46
Validation auc = 0.746009
Epoch 47
Validation auc = 0.707981
Epoch 48
Validation auc = 0.713615
Epoch 49
Loss = 3.5882e-01, PNorm = 55.8614, GNorm = 2.3234, lr_0 = 9.3122e-04
Validation auc = 0.721596
Epoch 50
Validation auc = 0.729577
Epoch 51
Validation auc = 0.730986
Epoch 52
Validation auc = 0.729577
Epoch 53
Validation auc = 0.728638
Epoch 54
Loss = 3.1873e-01, PNorm = 56.0848, GNorm = 0.9239, lr_0 = 9.2326e-04
Validation auc = 0.723944
Epoch 55
Validation auc = 0.723005
Epoch 56
Validation auc = 0.721596
Epoch 57
Validation auc = 0.715962
Epoch 58
Validation auc = 0.715493
Epoch 59
Loss = 2.0950e-01, PNorm = 56.3138, GNorm = 1.0270, lr_0 = 9.1537e-04
Validation auc = 0.722066
Epoch 60
Validation auc = 0.710798
Epoch 61
Validation auc = 0.702817
Epoch 62
Validation auc = 0.690610
Epoch 63
Validation auc = 0.698122
Epoch 64
Loss = 2.9695e-01, PNorm = 56.5722, GNorm = 1.6889, lr_0 = 9.0754e-04
Validation auc = 0.701408
Epoch 65
Validation auc = 0.674648
Epoch 66
Validation auc = 0.685915
Epoch 67
Validation auc = 0.684507
Epoch 68
Validation auc = 0.702347
Epoch 69
Loss = 1.0928e-01, PNorm = 56.8160, GNorm = 0.9883, lr_0 = 8.9978e-04
Validation auc = 0.731455
Epoch 70
Validation auc = 0.705634
Epoch 71
Validation auc = 0.703286
Epoch 72
Validation auc = 0.710329
Epoch 73
Validation auc = 0.676526
Epoch 74
Loss = 1.6047e-01, PNorm = 57.0331, GNorm = 1.7814, lr_0 = 8.9209e-04
Validation auc = 0.691549
Epoch 75
Validation auc = 0.722535
Epoch 76
Validation auc = 0.714554
Epoch 77
Validation auc = 0.702347
Epoch 78
Validation auc = 0.700939
Epoch 79
Loss = 9.6519e-02, PNorm = 57.2566, GNorm = 0.8732, lr_0 = 8.8447e-04
Validation auc = 0.702347
Epoch 80
Validation auc = 0.718779
Epoch 81
Validation auc = 0.702347
Epoch 82
Validation auc = 0.698592
Epoch 83
Validation auc = 0.709390
Epoch 84
Loss = 1.1335e-01, PNorm = 57.4485, GNorm = 0.8792, lr_0 = 8.7691e-04
Validation auc = 0.707981
Epoch 85
Validation auc = 0.715023
Epoch 86
Validation auc = 0.728638
Epoch 87
Validation auc = 0.732394
Epoch 88
Validation auc = 0.708920
Epoch 89
Loss = 1.0666e-01, PNorm = 57.6236, GNorm = 0.6390, lr_0 = 8.6941e-04
Validation auc = 0.705164
Epoch 90
Validation auc = 0.712207
Epoch 91
Validation auc = 0.704695
Epoch 92
Validation auc = 0.722535
Epoch 93
Validation auc = 0.718310
Epoch 94
Loss = 1.0827e-01, PNorm = 57.7888, GNorm = 0.8780, lr_0 = 8.6198e-04
Validation auc = 0.697183
Epoch 95
Validation auc = 0.709859
Epoch 96
Validation auc = 0.693897
Epoch 97
Validation auc = 0.686854
Epoch 98
Validation auc = 0.716901
Epoch 99
Loss = 1.7369e-01, PNorm = 57.9649, GNorm = 4.9833, lr_0 = 8.5461e-04
Validation auc = 0.661502
Epoch 100
Validation auc = 0.653991
Epoch 101
Validation auc = 0.684507
Epoch 102
Validation auc = 0.696714
Epoch 103
Validation auc = 0.705634
Epoch 104
Loss = 6.5573e-02, PNorm = 58.2201, GNorm = 0.6287, lr_0 = 8.4730e-04
Validation auc = 0.716901
Epoch 105
Validation auc = 0.729577
Epoch 106
Validation auc = 0.737559
Epoch 107
Validation auc = 0.740376
Epoch 108
Validation auc = 0.736620
Epoch 109
Loss = 1.1558e-01, PNorm = 58.4698, GNorm = 0.9108, lr_0 = 8.4006e-04
Validation auc = 0.737089
Epoch 110
Validation auc = 0.748826
Epoch 111
Validation auc = 0.742254
Epoch 112
Validation auc = 0.739437
Epoch 113
Validation auc = 0.744601
Epoch 114
Loss = 6.1822e-02, PNorm = 58.6538, GNorm = 0.6157, lr_0 = 8.3288e-04
Validation auc = 0.751643
Epoch 115
Validation auc = 0.752113
Epoch 116
Validation auc = 0.743662
Epoch 117
Validation auc = 0.732394
Epoch 118
Validation auc = 0.730047
Epoch 119
Loss = 3.4920e-02, PNorm = 58.7956, GNorm = 0.7897, lr_0 = 8.2576e-04
Validation auc = 0.740376
Epoch 120
Validation auc = 0.746948
Epoch 121
Validation auc = 0.732864
Epoch 122
Validation auc = 0.723474
Epoch 123
Validation auc = 0.737559
Epoch 124
Loss = 1.0766e-01, PNorm = 58.9141, GNorm = 0.8935, lr_0 = 8.1870e-04
Validation auc = 0.737089
Epoch 125
Validation auc = 0.721596
Epoch 126
Validation auc = 0.699531
Epoch 127
Validation auc = 0.691080
Epoch 128
Validation auc = 0.692019
Epoch 129
Loss = 8.6346e-02, PNorm = 59.0400, GNorm = 0.6807, lr_0 = 8.1170e-04
Validation auc = 0.689202
Epoch 130
Validation auc = 0.697183
Epoch 131
Validation auc = 0.713146
Epoch 132
Validation auc = 0.715493
Epoch 133
Validation auc = 0.715962
Epoch 134
Loss = 1.7370e-02, PNorm = 59.1702, GNorm = 0.2832, lr_0 = 8.0476e-04
Validation auc = 0.703756
Epoch 135
Validation auc = 0.711737
Epoch 136
Validation auc = 0.725352
Epoch 137
Validation auc = 0.724413
Epoch 138
Validation auc = 0.723474
Epoch 139
Loss = 6.2931e-02, PNorm = 59.2829, GNorm = 0.8750, lr_0 = 7.9788e-04
Validation auc = 0.730516
Epoch 140
Validation auc = 0.717840
Epoch 141
Validation auc = 0.706573
Epoch 142
Validation auc = 0.722066
Epoch 143
Validation auc = 0.736620
Epoch 144
Loss = 6.4957e-02, PNorm = 59.4026, GNorm = 0.3928, lr_0 = 7.9106e-04
Validation auc = 0.725822
Epoch 145
Validation auc = 0.702817
Epoch 146
Validation auc = 0.695775
Epoch 147
Validation auc = 0.696244
Epoch 148
Validation auc = 0.703756
Epoch 149
Loss = 1.3092e-02, PNorm = 59.5192, GNorm = 0.4478, lr_0 = 7.8430e-04
Validation auc = 0.704695
Epoch 150
Validation auc = 0.716432
Epoch 151
Validation auc = 0.720657
Epoch 152
Validation auc = 0.710798
Epoch 153
Validation auc = 0.707981
Epoch 154
Loss = 1.9122e-02, PNorm = 59.6159, GNorm = 0.2004, lr_0 = 7.7759e-04
Validation auc = 0.708920
Epoch 155
Validation auc = 0.712676
Epoch 156
Validation auc = 0.717840
Epoch 157
Validation auc = 0.711737
Epoch 158
Validation auc = 0.700469
Epoch 159
Loss = 2.2242e-02, PNorm = 59.7108, GNorm = 0.6099, lr_0 = 7.7095e-04
Validation auc = 0.712676
Epoch 160
Validation auc = 0.724413
Epoch 161
Validation auc = 0.722535
Epoch 162
Validation auc = 0.721127
Epoch 163
Validation auc = 0.725352
Epoch 164
Loss = 7.8634e-03, PNorm = 59.7978, GNorm = 0.1688, lr_0 = 7.6436e-04
Validation auc = 0.726761
Epoch 165
Validation auc = 0.727230
Epoch 166
Validation auc = 0.716432
Epoch 167
Validation auc = 0.702347
Epoch 168
Validation auc = 0.691549
Epoch 169
Loss = 3.4359e-03, PNorm = 59.8702, GNorm = 0.0845, lr_0 = 7.5782e-04
Validation auc = 0.680282
Epoch 170
Validation auc = 0.678404
Epoch 171
Validation auc = 0.681221
Epoch 172
Validation auc = 0.696714
Epoch 173
Validation auc = 0.704225
Epoch 174
Loss = 2.2343e-02, PNorm = 59.9475, GNorm = 1.2598, lr_0 = 7.5134e-04
Validation auc = 0.701408
Epoch 175
Validation auc = 0.702817
Epoch 176
Validation auc = 0.709859
Epoch 177
Validation auc = 0.715023
Epoch 178
Validation auc = 0.714085
Epoch 179
Loss = 7.3334e-03, PNorm = 60.0347, GNorm = 0.1249, lr_0 = 7.4492e-04
Validation auc = 0.707981
Epoch 180
Validation auc = 0.702347
Epoch 181
Validation auc = 0.707981
Epoch 182
Validation auc = 0.711268
Epoch 183
Validation auc = 0.715023
Epoch 184
Loss = 1.1028e-02, PNorm = 60.1168, GNorm = 0.1447, lr_0 = 7.3855e-04
Validation auc = 0.712207
Epoch 185
Validation auc = 0.715023
Epoch 186
Validation auc = 0.712207
Epoch 187
Validation auc = 0.714554
Epoch 188
Validation auc = 0.715493
Epoch 189
Loss = 8.5695e-03, PNorm = 60.1745, GNorm = 0.5396, lr_0 = 7.3224e-04
Validation auc = 0.713146
Epoch 190
Validation auc = 0.712676
Epoch 191
Validation auc = 0.707981
Epoch 192
Validation auc = 0.707042
Epoch 193
Validation auc = 0.710798
Epoch 194
Loss = 4.4191e-03, PNorm = 60.2209, GNorm = 0.2129, lr_0 = 7.2598e-04
Validation auc = 0.708920
Epoch 195
Validation auc = 0.691080
Epoch 196
Validation auc = 0.676291
Epoch 197
Validation auc = 0.682629
Epoch 198
Validation auc = 0.700000
Epoch 199
Loss = 3.1494e-02, PNorm = 60.3044, GNorm = 0.9734, lr_0 = 7.1977e-04
Validation auc = 0.703286
Epoch 200
Validation auc = 0.699061
Epoch 201
Validation auc = 0.700000
Epoch 202
Validation auc = 0.704695
Epoch 203
Validation auc = 0.708451
Epoch 204
Loss = 7.8306e-03, PNorm = 60.4157, GNorm = 0.0733, lr_0 = 7.1362e-04
Validation auc = 0.711737
Epoch 205
Validation auc = 0.714085
Epoch 206
Validation auc = 0.715493
Epoch 207
Validation auc = 0.713615
Epoch 208
Validation auc = 0.710798
Epoch 209
Loss = 5.2663e-03, PNorm = 60.4996, GNorm = 0.2198, lr_0 = 7.0752e-04
Validation auc = 0.707512
Epoch 210
Validation auc = 0.701408
Epoch 211
Validation auc = 0.698592
Epoch 212
Validation auc = 0.695305
Epoch 213
Validation auc = 0.695775
Epoch 214
Loss = 1.7229e-03, PNorm = 60.5539, GNorm = 0.0594, lr_0 = 7.0147e-04
Validation auc = 0.696714
Epoch 215
Validation auc = 0.698122
Epoch 216
Validation auc = 0.698592
Epoch 217
Validation auc = 0.700939
Epoch 218
Validation auc = 0.701878
Epoch 219
Loss = 1.7346e-03, PNorm = 60.5844, GNorm = 0.0784, lr_0 = 6.9548e-04
Validation auc = 0.700000
Epoch 220
Validation auc = 0.699531
Epoch 221
Validation auc = 0.704225
Epoch 222
Validation auc = 0.711737
Epoch 223
Validation auc = 0.715493
Epoch 224
Loss = 1.0396e-02, PNorm = 60.6111, GNorm = 0.5730, lr_0 = 6.8953e-04
Validation auc = 0.714085
Epoch 225
Validation auc = 0.712207
Epoch 226
Validation auc = 0.714554
Epoch 227
Validation auc = 0.708451
Epoch 228
Validation auc = 0.709390
Epoch 229
Loss = 8.3911e-04, PNorm = 60.6421, GNorm = 0.0611, lr_0 = 6.8364e-04
Validation auc = 0.711268
Epoch 230
Validation auc = 0.712676
Epoch 231
Validation auc = 0.714085
Epoch 232
Validation auc = 0.707512
Epoch 233
Validation auc = 0.700000
Epoch 234
Loss = 4.2770e-03, PNorm = 60.6790, GNorm = 0.5375, lr_0 = 6.7779e-04
Validation auc = 0.699061
Epoch 235
Validation auc = 0.699061
Epoch 236
Validation auc = 0.703286
Epoch 237
Validation auc = 0.704225
Epoch 238
Validation auc = 0.707042
Epoch 239
Loss = 9.6459e-04, PNorm = 60.7141, GNorm = 0.0071, lr_0 = 6.7200e-04
Validation auc = 0.711737
Epoch 240
Validation auc = 0.715962
Epoch 241
Validation auc = 0.718779
Epoch 242
Validation auc = 0.720657
Epoch 243
Validation auc = 0.716901
Epoch 244
Loss = 3.2438e-03, PNorm = 60.7424, GNorm = 0.1630, lr_0 = 6.6625e-04
Validation auc = 0.718310
Epoch 245
Validation auc = 0.716901
Epoch 246
Validation auc = 0.719249
Epoch 247
Validation auc = 0.717371
Epoch 248
Validation auc = 0.713146
Epoch 249
Loss = 1.8571e-03, PNorm = 60.7756, GNorm = 0.0880, lr_0 = 6.6056e-04
Validation auc = 0.710798
Epoch 250
Validation auc = 0.709859
Epoch 251
Validation auc = 0.707512
Epoch 252
Validation auc = 0.705634
Epoch 253
Validation auc = 0.703756
Epoch 254
Loss = 5.6035e-04, PNorm = 60.8110, GNorm = 0.0583, lr_0 = 6.5491e-04
Validation auc = 0.701878
Epoch 255
Validation auc = 0.712676
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.720188
Epoch 258
Validation auc = 0.716901
Epoch 259
Loss = 2.5144e-03, PNorm = 60.8674, GNorm = 0.1903, lr_0 = 6.4931e-04
Validation auc = 0.711737
Epoch 260
Validation auc = 0.707981
Epoch 261
Validation auc = 0.707981
Epoch 262
Validation auc = 0.710798
Epoch 263
Validation auc = 0.709859
Epoch 264
Loss = 2.2762e-02, PNorm = 60.9220, GNorm = 1.3346, lr_0 = 6.4376e-04
Validation auc = 0.699531
Epoch 265
Validation auc = 0.689671
Epoch 266
Validation auc = 0.676056
Epoch 267
Validation auc = 0.667606
Epoch 268
Validation auc = 0.673709
Epoch 269
Loss = 1.5829e-03, PNorm = 60.9870, GNorm = 0.0673, lr_0 = 6.3826e-04
Validation auc = 0.680282
Epoch 270
Validation auc = 0.683099
Epoch 271
Validation auc = 0.684038
Epoch 272
Validation auc = 0.685915
Epoch 273
Validation auc = 0.684977
Epoch 274
Loss = 1.7485e-02, PNorm = 61.0390, GNorm = 1.2095, lr_0 = 6.3280e-04
Validation auc = 0.686385
Epoch 275
Validation auc = 0.688732
Epoch 276
Validation auc = 0.690610
Epoch 277
Validation auc = 0.690610
Epoch 278
Validation auc = 0.689202
Epoch 279
Loss = 1.1717e-03, PNorm = 61.0774, GNorm = 0.0725, lr_0 = 6.2739e-04
Validation auc = 0.687324
Epoch 280
Validation auc = 0.684977
Epoch 281
Validation auc = 0.682160
Epoch 282
Validation auc = 0.680751
Epoch 283
Validation auc = 0.684038
Epoch 284
Loss = 1.1472e-03, PNorm = 61.1123, GNorm = 0.0417, lr_0 = 6.2203e-04
Validation auc = 0.688263
Epoch 285
Validation auc = 0.689202
Epoch 286
Validation auc = 0.688263
Epoch 287
Validation auc = 0.688732
Epoch 288
Validation auc = 0.690610
Epoch 289
Loss = 6.0798e-04, PNorm = 61.1334, GNorm = 0.0327, lr_0 = 6.1671e-04
Validation auc = 0.689671
Epoch 290
Validation auc = 0.690610
Epoch 291
Validation auc = 0.692019
Epoch 292
Validation auc = 0.692019
Epoch 293
Validation auc = 0.692958
Epoch 294
Loss = 1.7094e-03, PNorm = 61.1508, GNorm = 0.0255, lr_0 = 6.1144e-04
Validation auc = 0.692958
Epoch 295
Validation auc = 0.692958
Epoch 296
Validation auc = 0.692958
Epoch 297
Validation auc = 0.690610
Epoch 298
Validation auc = 0.690610
Epoch 299
Loss = 1.6124e-03, PNorm = 61.1620, GNorm = 0.1471, lr_0 = 6.0621e-04
Validation auc = 0.690141
Model 0 best validation auc = 0.752113 on epoch 115
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.838049
Ensemble test auc = 0.838049
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.449296
Epoch 1
Validation auc = 0.480751
Epoch 2
Validation auc = 0.507512
Epoch 3
Validation auc = 0.558216
Epoch 4
Loss = 7.0163e-01, PNorm = 54.1605, GNorm = 1.6551, lr_0 = 6.5000e-04
Validation auc = 0.590141
Epoch 5
Validation auc = 0.636620
Epoch 6
Validation auc = 0.645070
Epoch 7
Validation auc = 0.643662
Epoch 8
Validation auc = 0.671362
Epoch 9
Loss = 5.5033e-01, PNorm = 54.3091, GNorm = 1.3546, lr_0 = 9.9743e-04
Validation auc = 0.682160
Epoch 10
Validation auc = 0.679812
Epoch 11
Validation auc = 0.685915
Epoch 12
Validation auc = 0.684977
Epoch 13
Validation auc = 0.695775
Epoch 14
Loss = 4.3778e-01, PNorm = 54.5068, GNorm = 0.9656, lr_0 = 9.8890e-04
Validation auc = 0.702347
Epoch 15
Validation auc = 0.725352
Epoch 16
Validation auc = 0.699531
Epoch 17
Validation auc = 0.699531
Epoch 18
Validation auc = 0.696714
Epoch 19
Loss = 5.1419e-01, PNorm = 54.6625, GNorm = 1.1759, lr_0 = 9.8045e-04
Validation auc = 0.692488
Epoch 20
Validation auc = 0.694366
Epoch 21
Validation auc = 0.694836
Epoch 22
Validation auc = 0.699531
Epoch 23
Validation auc = 0.710329
Epoch 24
Loss = 3.9901e-01, PNorm = 54.8369, GNorm = 1.1448, lr_0 = 9.7207e-04
Validation auc = 0.717840
Epoch 25
Validation auc = 0.721596
Epoch 26
Validation auc = 0.730516
Epoch 27
Validation auc = 0.734742
Epoch 28
Validation auc = 0.729108
Epoch 29
Loss = 4.0922e-01, PNorm = 54.9994, GNorm = 0.7266, lr_0 = 9.6376e-04
Validation auc = 0.726291
Epoch 30
Validation auc = 0.727700
Epoch 31
Validation auc = 0.717371
Epoch 32
Validation auc = 0.708920
Epoch 33
Validation auc = 0.709390
Epoch 34
Loss = 4.3170e-01, PNorm = 55.1794, GNorm = 0.9177, lr_0 = 9.5552e-04
Validation auc = 0.705164
Epoch 35
Validation auc = 0.711268
Epoch 36
Validation auc = 0.713146
Epoch 37
Validation auc = 0.712207
Epoch 38
Validation auc = 0.709859
Epoch 39
Loss = 3.1579e-01, PNorm = 55.3316, GNorm = 0.8132, lr_0 = 9.4735e-04
Validation auc = 0.707042
Epoch 40
Validation auc = 0.705164
Epoch 41
Validation auc = 0.698122
Epoch 42
Validation auc = 0.697653
Epoch 43
Validation auc = 0.691549
Epoch 44
Loss = 3.2706e-01, PNorm = 55.5066, GNorm = 1.0059, lr_0 = 9.3925e-04
Validation auc = 0.701408
Epoch 45
Validation auc = 0.702347
Epoch 46
Validation auc = 0.700469
Epoch 47
Validation auc = 0.705634
Epoch 48
Validation auc = 0.708920
Epoch 49
Loss = 2.7474e-01, PNorm = 55.6958, GNorm = 0.5486, lr_0 = 9.3122e-04
Validation auc = 0.701878
Epoch 50
Validation auc = 0.692488
Epoch 51
Validation auc = 0.682160
Epoch 52
Validation auc = 0.684977
Epoch 53
Validation auc = 0.665258
Epoch 54
Loss = 3.4332e-01, PNorm = 55.8974, GNorm = 2.1588, lr_0 = 9.2326e-04
Validation auc = 0.670423
Epoch 55
Validation auc = 0.676056
Epoch 56
Validation auc = 0.654930
Epoch 57
Validation auc = 0.674648
Epoch 58
Validation auc = 0.679812
Epoch 59
Loss = 2.8029e-01, PNorm = 56.1235, GNorm = 1.2780, lr_0 = 9.1537e-04
Validation auc = 0.670423
Epoch 60
Validation auc = 0.666667
Epoch 61
Validation auc = 0.681221
Epoch 62
Validation auc = 0.670423
Epoch 63
Validation auc = 0.676995
Epoch 64
Loss = 1.3902e-01, PNorm = 56.3839, GNorm = 1.5392, lr_0 = 9.0754e-04
Validation auc = 0.685915
Epoch 65
Validation auc = 0.689671
Epoch 66
Validation auc = 0.677934
Epoch 67
Validation auc = 0.669484
Epoch 68
Validation auc = 0.674648
Epoch 69
Loss = 1.9405e-01, PNorm = 56.6526, GNorm = 3.7422, lr_0 = 8.9978e-04
Validation auc = 0.694836
Epoch 70
Validation auc = 0.660563
Epoch 71
Validation auc = 0.669014
Epoch 72
Validation auc = 0.669953
Epoch 73
Validation auc = 0.668545
Epoch 74
Loss = 1.6343e-01, PNorm = 56.8985, GNorm = 1.2042, lr_0 = 8.9209e-04
Validation auc = 0.681221
Epoch 75
Validation auc = 0.675587
Epoch 76
Validation auc = 0.680282
Epoch 77
Validation auc = 0.688732
Epoch 78
Validation auc = 0.671831
Epoch 79
Loss = 1.5234e-01, PNorm = 57.1611, GNorm = 0.7566, lr_0 = 8.8447e-04
Validation auc = 0.656808
Epoch 80
Validation auc = 0.659624
Epoch 81
Validation auc = 0.647887
Epoch 82
Validation auc = 0.659155
Epoch 83
Validation auc = 0.654930
Epoch 84
Loss = 2.1742e-01, PNorm = 57.4000, GNorm = 1.4399, lr_0 = 8.7691e-04
Validation auc = 0.682629
Epoch 85
Validation auc = 0.666197
Epoch 86
Validation auc = 0.664319
Epoch 87
Validation auc = 0.685446
Epoch 88
Validation auc = 0.657746
Epoch 89
Loss = 8.7449e-02, PNorm = 57.6157, GNorm = 1.7863, lr_0 = 8.6941e-04
Validation auc = 0.631925
Epoch 90
Validation auc = 0.649296
Epoch 91
Validation auc = 0.659624
Epoch 92
Validation auc = 0.667136
Epoch 93
Validation auc = 0.678404
Epoch 94
Loss = 7.9415e-02, PNorm = 57.8373, GNorm = 1.9281, lr_0 = 8.6198e-04
Validation auc = 0.700469
Epoch 95
Validation auc = 0.717371
Epoch 96
Validation auc = 0.689671
Epoch 97
Validation auc = 0.680282
Epoch 98
Validation auc = 0.676056
Epoch 99
Loss = 1.0172e-01, PNorm = 58.0371, GNorm = 0.6060, lr_0 = 8.5461e-04
Validation auc = 0.651174
Epoch 100
Validation auc = 0.655869
Epoch 101
Validation auc = 0.677465
Epoch 102
Validation auc = 0.690610
Epoch 103
Validation auc = 0.673709
Epoch 104
Loss = 5.7465e-02, PNorm = 58.2172, GNorm = 0.8754, lr_0 = 8.4730e-04
Validation auc = 0.676056
Epoch 105
Validation auc = 0.687324
Epoch 106
Validation auc = 0.691549
Epoch 107
Validation auc = 0.662441
Epoch 108
Validation auc = 0.661502
Epoch 109
Loss = 2.8606e-02, PNorm = 58.3674, GNorm = 0.4459, lr_0 = 8.4006e-04
Validation auc = 0.664319
Epoch 110
Validation auc = 0.661972
Epoch 111
Validation auc = 0.665258
Epoch 112
Validation auc = 0.688732
Epoch 113
Validation auc = 0.692019
Epoch 114
Loss = 5.5714e-02, PNorm = 58.5042, GNorm = 0.8377, lr_0 = 8.3288e-04
Validation auc = 0.699531
Epoch 115
Validation auc = 0.697653
Epoch 116
Validation auc = 0.688732
Epoch 117
Validation auc = 0.679812
Epoch 118
Validation auc = 0.678404
Epoch 119
Loss = 1.0663e-01, PNorm = 58.6166, GNorm = 4.7585, lr_0 = 8.2576e-04
Validation auc = 0.657277
Epoch 120
Validation auc = 0.642723
Epoch 121
Validation auc = 0.660094
Epoch 122
Validation auc = 0.672300
Epoch 123
Validation auc = 0.669953
Epoch 124
Loss = 8.3376e-02, PNorm = 58.7502, GNorm = 0.8572, lr_0 = 8.1870e-04
Validation auc = 0.674178
Epoch 125
Validation auc = 0.680282
Epoch 126
Validation auc = 0.683099
Epoch 127
Validation auc = 0.672770
Epoch 128
Validation auc = 0.655869
Epoch 129
Loss = 3.2299e-02, PNorm = 58.8970, GNorm = 0.5573, lr_0 = 8.1170e-04
Validation auc = 0.647887
Epoch 130
Validation auc = 0.643662
Epoch 131
Validation auc = 0.644131
Epoch 132
Validation auc = 0.635681
Epoch 133
Validation auc = 0.636150
Epoch 134
Loss = 6.8729e-02, PNorm = 59.0260, GNorm = 1.1028, lr_0 = 8.0476e-04
Validation auc = 0.647887
Epoch 135
Validation auc = 0.657746
Epoch 136
Validation auc = 0.659624
Epoch 137
Validation auc = 0.653521
Epoch 138
Validation auc = 0.655399
Epoch 139
Loss = 9.7363e-02, PNorm = 59.1229, GNorm = 0.5224, lr_0 = 7.9788e-04
Validation auc = 0.668545
Epoch 140
Validation auc = 0.654930
Epoch 141
Validation auc = 0.658685
Epoch 142
Validation auc = 0.677934
Epoch 143
Validation auc = 0.688263
Epoch 144
Loss = 6.1594e-02, PNorm = 59.1999, GNorm = 2.8424, lr_0 = 7.9106e-04
Validation auc = 0.679812
Epoch 145
Validation auc = 0.649765
Epoch 146
Validation auc = 0.647887
Epoch 147
Validation auc = 0.647418
Epoch 148
Validation auc = 0.643662
Epoch 149
Loss = 3.4260e-02, PNorm = 59.2978, GNorm = 0.1305, lr_0 = 7.8430e-04
Validation auc = 0.642723
Epoch 150
Validation auc = 0.645070
Epoch 151
Validation auc = 0.644601
Epoch 152
Validation auc = 0.636150
Epoch 153
Validation auc = 0.629577
Epoch 154
Loss = 6.9804e-02, PNorm = 59.3918, GNorm = 2.1389, lr_0 = 7.7759e-04
Validation auc = 0.631455
Epoch 155
Validation auc = 0.645540
Epoch 156
Validation auc = 0.652113
Epoch 157
Validation auc = 0.650235
Epoch 158
Validation auc = 0.670423
Epoch 159
Loss = 4.3641e-02, PNorm = 59.5157, GNorm = 1.2863, lr_0 = 7.7095e-04
Validation auc = 0.695775
Epoch 160
Validation auc = 0.690141
Epoch 161
Validation auc = 0.684038
Epoch 162
Validation auc = 0.680751
Epoch 163
Validation auc = 0.675117
Epoch 164
Loss = 1.3452e-02, PNorm = 59.6397, GNorm = 0.2277, lr_0 = 7.6436e-04
Validation auc = 0.665258
Epoch 165
Validation auc = 0.661502
Epoch 166
Validation auc = 0.686385
Epoch 167
Validation auc = 0.694836
Epoch 168
Validation auc = 0.669014
Epoch 169
Loss = 5.3398e-02, PNorm = 59.7517, GNorm = 0.4621, lr_0 = 7.5782e-04
Validation auc = 0.684977
Epoch 170
Validation auc = 0.696244
Epoch 171
Validation auc = 0.668075
Epoch 172
Validation auc = 0.633803
Epoch 173
Validation auc = 0.646948
Epoch 174
Loss = 8.9180e-02, PNorm = 59.8634, GNorm = 0.7069, lr_0 = 7.5134e-04
Validation auc = 0.669484
Epoch 175
Validation auc = 0.666197
Epoch 176
Validation auc = 0.659155
Epoch 177
Validation auc = 0.647418
Epoch 178
Validation auc = 0.647418
Epoch 179
Loss = 1.9900e-02, PNorm = 60.0041, GNorm = 0.2897, lr_0 = 7.4492e-04
Validation auc = 0.649296
Epoch 180
Validation auc = 0.657746
Epoch 181
Validation auc = 0.655399
Epoch 182
Validation auc = 0.653052
Epoch 183
Validation auc = 0.650235
Epoch 184
Loss = 4.4540e-03, PNorm = 60.1180, GNorm = 0.0584, lr_0 = 7.3855e-04
Validation auc = 0.653052
Epoch 185
Validation auc = 0.657746
Epoch 186
Validation auc = 0.660563
Epoch 187
Validation auc = 0.656808
Epoch 188
Validation auc = 0.653991
Epoch 189
Loss = 2.2360e-03, PNorm = 60.2016, GNorm = 0.0223, lr_0 = 7.3224e-04
Validation auc = 0.651643
Epoch 190
Validation auc = 0.650235
Epoch 191
Validation auc = 0.654930
Epoch 192
Validation auc = 0.659155
Epoch 193
Validation auc = 0.655399
Epoch 194
Loss = 9.4752e-03, PNorm = 60.2802, GNorm = 0.5122, lr_0 = 7.2598e-04
Validation auc = 0.651643
Epoch 195
Validation auc = 0.648357
Epoch 196
Validation auc = 0.643662
Epoch 197
Validation auc = 0.649296
Epoch 198
Validation auc = 0.649296
Epoch 199
Loss = 2.0693e-02, PNorm = 60.3488, GNorm = 0.6787, lr_0 = 7.1977e-04
Validation auc = 0.648357
Epoch 200
Validation auc = 0.641784
Epoch 201
Validation auc = 0.639437
Epoch 202
Validation auc = 0.646479
Epoch 203
Validation auc = 0.653521
Epoch 204
Loss = 2.2135e-03, PNorm = 60.4115, GNorm = 0.0516, lr_0 = 7.1362e-04
Validation auc = 0.657277
Epoch 205
Validation auc = 0.659624
Epoch 206
Validation auc = 0.659155
Epoch 207
Validation auc = 0.658216
Epoch 208
Validation auc = 0.656338
Epoch 209
Loss = 1.1978e-03, PNorm = 60.4504, GNorm = 0.0229, lr_0 = 7.0752e-04
Validation auc = 0.654930
Epoch 210
Validation auc = 0.654460
Epoch 211
Validation auc = 0.656338
Epoch 212
Validation auc = 0.656338
Epoch 213
Validation auc = 0.652113
Epoch 214
Loss = 6.8851e-02, PNorm = 60.4788, GNorm = 0.1304, lr_0 = 7.0147e-04
Validation auc = 0.647418
Epoch 215
Validation auc = 0.648357
Epoch 216
Validation auc = 0.646948
Epoch 217
Validation auc = 0.650704
Epoch 218
Validation auc = 0.654930
Epoch 219
Loss = 4.7511e-03, PNorm = 60.5008, GNorm = 0.1522, lr_0 = 6.9548e-04
Validation auc = 0.659155
Epoch 220
Validation auc = 0.661502
Epoch 221
Validation auc = 0.661502
Epoch 222
Validation auc = 0.657746
Epoch 223
Validation auc = 0.658685
Epoch 224
Loss = 1.7147e-02, PNorm = 60.5274, GNorm = 0.8381, lr_0 = 6.8953e-04
Validation auc = 0.657277
Epoch 225
Validation auc = 0.657277
Epoch 226
Validation auc = 0.661972
Epoch 227
Validation auc = 0.664789
Epoch 228
Validation auc = 0.672300
Epoch 229
Loss = 5.5891e-03, PNorm = 60.5539, GNorm = 0.1466, lr_0 = 6.8364e-04
Validation auc = 0.671831
Epoch 230
Validation auc = 0.676056
Epoch 231
Validation auc = 0.674648
Epoch 232
Validation auc = 0.676995
Epoch 233
Validation auc = 0.677934
Epoch 234
Loss = 3.9813e-03, PNorm = 60.5843, GNorm = 0.1474, lr_0 = 6.7779e-04
Validation auc = 0.676526
Epoch 235
Validation auc = 0.672300
Epoch 236
Validation auc = 0.669014
Epoch 237
Validation auc = 0.669484
Epoch 238
Validation auc = 0.671831
Epoch 239
Loss = 3.6707e-03, PNorm = 60.6081, GNorm = 0.1519, lr_0 = 6.7200e-04
Validation auc = 0.676056
Epoch 240
Validation auc = 0.666197
Epoch 241
Validation auc = 0.662911
Epoch 242
Validation auc = 0.667136
Epoch 243
Validation auc = 0.669014
Epoch 244
Loss = 7.2588e-03, PNorm = 60.6361, GNorm = 0.2400, lr_0 = 6.6625e-04
Validation auc = 0.675587
Epoch 245
Validation auc = 0.678404
Epoch 246
Validation auc = 0.676526
Epoch 247
Validation auc = 0.676526
Epoch 248
Validation auc = 0.673239
Epoch 249
Loss = 1.1414e-02, PNorm = 60.6678, GNorm = 0.0124, lr_0 = 6.6056e-04
Validation auc = 0.673709
Epoch 250
Validation auc = 0.673709
Epoch 251
Validation auc = 0.673709
Epoch 252
Validation auc = 0.675587
Epoch 253
Validation auc = 0.674178
Epoch 254
Loss = 1.2801e-03, PNorm = 60.7018, GNorm = 0.0251, lr_0 = 6.5491e-04
Validation auc = 0.675587
Epoch 255
Validation auc = 0.674648
Epoch 256
Validation auc = 0.680751
Epoch 257
Validation auc = 0.684038
Epoch 258
Validation auc = 0.678873
Epoch 259
Loss = 3.5217e-03, PNorm = 60.7322, GNorm = 0.0111, lr_0 = 6.4931e-04
Validation auc = 0.666667
Epoch 260
Validation auc = 0.655399
Epoch 261
Validation auc = 0.672300
Epoch 262
Validation auc = 0.686385
Epoch 263
Validation auc = 0.689671
Epoch 264
Loss = 7.1945e-03, PNorm = 60.7836, GNorm = 0.1164, lr_0 = 6.4376e-04
Validation auc = 0.685446
Epoch 265
Validation auc = 0.682629
Epoch 266
Validation auc = 0.684977
Epoch 267
Validation auc = 0.685446
Epoch 268
Validation auc = 0.684038
Epoch 269
Loss = 1.9324e-02, PNorm = 60.8373, GNorm = 0.1911, lr_0 = 6.3826e-04
Validation auc = 0.684038
Epoch 270
Validation auc = 0.672770
Epoch 271
Validation auc = 0.663380
Epoch 272
Validation auc = 0.660798
Epoch 273
Validation auc = 0.661972
Epoch 274
Loss = 3.8462e-03, PNorm = 60.8871, GNorm = 0.2927, lr_0 = 6.3280e-04
Validation auc = 0.676526
Epoch 275
Validation auc = 0.682629
Epoch 276
Validation auc = 0.683099
Epoch 277
Validation auc = 0.677934
Epoch 278
Validation auc = 0.671362
Epoch 279
Loss = 8.3536e-03, PNorm = 60.9323, GNorm = 0.0463, lr_0 = 6.2739e-04
Validation auc = 0.669953
Epoch 280
Validation auc = 0.665258
Epoch 281
Validation auc = 0.671362
Epoch 282
Validation auc = 0.675117
Epoch 283
Validation auc = 0.677934
Epoch 284
Loss = 1.4857e-03, PNorm = 60.9827, GNorm = 0.0813, lr_0 = 6.2203e-04
Validation auc = 0.676995
Epoch 285
Validation auc = 0.678873
Epoch 286
Validation auc = 0.680282
Epoch 287
Validation auc = 0.683568
Epoch 288
Validation auc = 0.684038
Epoch 289
Loss = 1.1777e-02, PNorm = 61.0150, GNorm = 0.0912, lr_0 = 6.1671e-04
Validation auc = 0.685446
Epoch 290
Validation auc = 0.684038
Epoch 291
Validation auc = 0.678404
Epoch 292
Validation auc = 0.676526
Epoch 293
Validation auc = 0.671831
Epoch 294
Loss = 1.1534e-03, PNorm = 61.0393, GNorm = 0.0751, lr_0 = 6.1144e-04
Validation auc = 0.668075
Epoch 295
Validation auc = 0.664789
Epoch 296
Validation auc = 0.662911
Epoch 297
Validation auc = 0.660563
Epoch 298
Validation auc = 0.657746
Epoch 299
Loss = 5.1355e-04, PNorm = 61.0577, GNorm = 0.0053, lr_0 = 6.0621e-04
Validation auc = 0.657277
Model 0 best validation auc = 0.734742 on epoch 27
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.893438
Ensemble test auc = 0.893438
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.459624
Epoch 1
Validation auc = 0.492958
Epoch 2
Validation auc = 0.512676
Epoch 3
Validation auc = 0.575587
Epoch 4
Loss = 6.5026e-01, PNorm = 54.1544, GNorm = 1.8066, lr_0 = 6.5000e-04
Validation auc = 0.601878
Epoch 5
Validation auc = 0.634742
Epoch 6
Validation auc = 0.638498
Epoch 7
Validation auc = 0.659155
Epoch 8
Validation auc = 0.666197
Epoch 9
Loss = 5.4745e-01, PNorm = 54.3094, GNorm = 1.3897, lr_0 = 9.9743e-04
Validation auc = 0.663850
Epoch 10
Validation auc = 0.675117
Epoch 11
Validation auc = 0.694836
Epoch 12
Validation auc = 0.714085
Epoch 13
Validation auc = 0.700469
Epoch 14
Loss = 4.6185e-01, PNorm = 54.4949, GNorm = 0.6867, lr_0 = 9.8890e-04
Validation auc = 0.692958
Epoch 15
Validation auc = 0.703756
Epoch 16
Validation auc = 0.714554
Epoch 17
Validation auc = 0.709859
Epoch 18
Validation auc = 0.716901
Epoch 19
Loss = 4.7169e-01, PNorm = 54.6765, GNorm = 0.8628, lr_0 = 9.8045e-04
Validation auc = 0.730986
Epoch 20
Validation auc = 0.729108
Epoch 21
Validation auc = 0.717371
Epoch 22
Validation auc = 0.731455
Epoch 23
Validation auc = 0.722535
Epoch 24
Loss = 5.0942e-01, PNorm = 54.8595, GNorm = 2.5227, lr_0 = 9.7207e-04
Validation auc = 0.707512
Epoch 25
Validation auc = 0.727230
Epoch 26
Validation auc = 0.722066
Epoch 27
Validation auc = 0.705634
Epoch 28
Validation auc = 0.706103
Epoch 29
Loss = 4.0043e-01, PNorm = 55.0593, GNorm = 1.0301, lr_0 = 9.6376e-04
Validation auc = 0.710798
Epoch 30
Validation auc = 0.720188
Epoch 31
Validation auc = 0.723944
Epoch 32
Validation auc = 0.727230
Epoch 33
Validation auc = 0.739906
Epoch 34
Loss = 3.9451e-01, PNorm = 55.2407, GNorm = 1.1926, lr_0 = 9.5552e-04
Validation auc = 0.728169
Epoch 35
Validation auc = 0.726291
Epoch 36
Validation auc = 0.730986
Epoch 37
Validation auc = 0.730047
Epoch 38
Validation auc = 0.727700
Epoch 39
Loss = 3.5199e-01, PNorm = 55.4167, GNorm = 0.6343, lr_0 = 9.4735e-04
Validation auc = 0.739906
Epoch 40
Validation auc = 0.737089
Epoch 41
Validation auc = 0.736150
Epoch 42
Validation auc = 0.729577
Epoch 43
Validation auc = 0.726761
Epoch 44
Loss = 3.1003e-01, PNorm = 55.6192, GNorm = 0.9349, lr_0 = 9.3925e-04
Validation auc = 0.702817
Epoch 45
Validation auc = 0.698592
Epoch 46
Validation auc = 0.697183
Epoch 47
Validation auc = 0.718779
Epoch 48
Validation auc = 0.723005
Epoch 49
Loss = 2.6491e-01, PNorm = 55.8471, GNorm = 1.1351, lr_0 = 9.3122e-04
Validation auc = 0.735211
Epoch 50
Validation auc = 0.718779
Epoch 51
Validation auc = 0.717371
Epoch 52
Validation auc = 0.699531
Epoch 53
Validation auc = 0.692019
Epoch 54
Loss = 2.0923e-01, PNorm = 56.0918, GNorm = 1.3095, lr_0 = 9.2326e-04
Validation auc = 0.690141
Epoch 55
Validation auc = 0.686854
Epoch 56
Validation auc = 0.689202
Epoch 57
Validation auc = 0.687324
Epoch 58
Validation auc = 0.692019
Epoch 59
Loss = 2.4926e-01, PNorm = 56.3565, GNorm = 3.4624, lr_0 = 9.1537e-04
Validation auc = 0.674648
Epoch 60
Validation auc = 0.689202
Epoch 61
Validation auc = 0.696714
Epoch 62
Validation auc = 0.704695
Epoch 63
Validation auc = 0.734272
Epoch 64
Loss = 2.1908e-01, PNorm = 56.6229, GNorm = 0.6309, lr_0 = 9.0754e-04
Validation auc = 0.701878
Epoch 65
Validation auc = 0.696714
Epoch 66
Validation auc = 0.700469
Epoch 67
Validation auc = 0.688263
Epoch 68
Validation auc = 0.680751
Epoch 69
Loss = 2.1660e-01, PNorm = 56.8958, GNorm = 0.6083, lr_0 = 8.9978e-04
Validation auc = 0.684038
Epoch 70
Validation auc = 0.663850
Epoch 71
Validation auc = 0.654930
Epoch 72
Validation auc = 0.664789
Epoch 73
Validation auc = 0.679343
Epoch 74
Loss = 1.3501e-01, PNorm = 57.1431, GNorm = 0.5448, lr_0 = 8.9209e-04
Validation auc = 0.679343
Epoch 75
Validation auc = 0.686854
Epoch 76
Validation auc = 0.697183
Epoch 77
Validation auc = 0.707981
Epoch 78
Validation auc = 0.686854
Epoch 79
Loss = 1.6344e-01, PNorm = 57.3891, GNorm = 1.6941, lr_0 = 8.8447e-04
Validation auc = 0.685915
Epoch 80
Validation auc = 0.698592
Epoch 81
Validation auc = 0.697183
Epoch 82
Validation auc = 0.689671
Epoch 83
Validation auc = 0.688263
Epoch 84
Loss = 5.5143e-02, PNorm = 57.6224, GNorm = 0.2548, lr_0 = 8.7691e-04
Validation auc = 0.694366
Epoch 85
Validation auc = 0.697653
Epoch 86
Validation auc = 0.692019
Epoch 87
Validation auc = 0.679812
Epoch 88
Validation auc = 0.694366
Epoch 89
Loss = 1.1417e-01, PNorm = 57.8192, GNorm = 1.9074, lr_0 = 8.6941e-04
Validation auc = 0.661972
Epoch 90
Validation auc = 0.657746
Epoch 91
Validation auc = 0.693427
Epoch 92
Validation auc = 0.707042
Epoch 93
Validation auc = 0.700939
Epoch 94
Loss = 1.6298e-01, PNorm = 58.0208, GNorm = 1.7855, lr_0 = 8.6198e-04
Validation auc = 0.690141
Epoch 95
Validation auc = 0.694836
Epoch 96
Validation auc = 0.683099
Epoch 97
Validation auc = 0.671831
Epoch 98
Validation auc = 0.673709
Epoch 99
Loss = 1.4666e-01, PNorm = 58.2360, GNorm = 0.9461, lr_0 = 8.5461e-04
Validation auc = 0.692958
Epoch 100
Validation auc = 0.688732
Epoch 101
Validation auc = 0.669484
Epoch 102
Validation auc = 0.659624
Epoch 103
Validation auc = 0.658216
Epoch 104
Loss = 3.9027e-02, PNorm = 58.4263, GNorm = 0.4776, lr_0 = 8.4730e-04
Validation auc = 0.660094
Epoch 105
Validation auc = 0.674178
Epoch 106
Validation auc = 0.691549
Epoch 107
Validation auc = 0.700000
Epoch 108
Validation auc = 0.693897
Epoch 109
Loss = 1.6267e-02, PNorm = 58.6123, GNorm = 0.2852, lr_0 = 8.4006e-04
Validation auc = 0.685446
Epoch 110
Validation auc = 0.694366
Epoch 111
Validation auc = 0.708920
Epoch 112
Validation auc = 0.672300
Epoch 113
Validation auc = 0.647418
Epoch 114
Loss = 3.6796e-01, PNorm = 58.7628, GNorm = 0.4916, lr_0 = 8.3288e-04
Validation auc = 0.671831
Epoch 115
Validation auc = 0.662911
Epoch 116
Validation auc = 0.661972
Epoch 117
Validation auc = 0.661033
Epoch 118
Validation auc = 0.667606
Epoch 119
Loss = 1.1572e-01, PNorm = 58.9670, GNorm = 0.7303, lr_0 = 8.2576e-04
Validation auc = 0.671362
Epoch 120
Validation auc = 0.661033
Epoch 121
Validation auc = 0.651643
Epoch 122
Validation auc = 0.642723
Epoch 123
Validation auc = 0.638967
Epoch 124
Loss = 5.7569e-02, PNorm = 59.1557, GNorm = 0.5776, lr_0 = 8.1870e-04
Validation auc = 0.639437
Epoch 125
Validation auc = 0.640845
Epoch 126
Validation auc = 0.646009
Epoch 127
Validation auc = 0.667136
Epoch 128
Validation auc = 0.686385
Epoch 129
Loss = 2.2564e-02, PNorm = 59.3180, GNorm = 0.3077, lr_0 = 8.1170e-04
Validation auc = 0.679343
Epoch 130
Validation auc = 0.683568
Epoch 131
Validation auc = 0.690610
Epoch 132
Validation auc = 0.697653
Epoch 133
Validation auc = 0.691549
Epoch 134
Loss = 1.8988e-02, PNorm = 59.4505, GNorm = 0.2609, lr_0 = 8.0476e-04
Validation auc = 0.690141
Epoch 135
Validation auc = 0.691549
Epoch 136
Validation auc = 0.691080
Epoch 137
Validation auc = 0.692488
Epoch 138
Validation auc = 0.701878
Epoch 139
Loss = 1.9056e-02, PNorm = 59.5563, GNorm = 0.3910, lr_0 = 7.9788e-04
Validation auc = 0.706103
Epoch 140
Validation auc = 0.701408
Epoch 141
Validation auc = 0.695305
Epoch 142
Validation auc = 0.694836
Epoch 143
Validation auc = 0.701878
Epoch 144
Loss = 1.8409e-02, PNorm = 59.6392, GNorm = 0.7701, lr_0 = 7.9106e-04
Validation auc = 0.702347
Epoch 145
Validation auc = 0.692958
Epoch 146
Validation auc = 0.683099
Epoch 147
Validation auc = 0.688263
Epoch 148
Validation auc = 0.692488
Epoch 149
Loss = 6.9158e-02, PNorm = 59.7134, GNorm = 1.1748, lr_0 = 7.8430e-04
Validation auc = 0.694366
Epoch 150
Validation auc = 0.693897
Epoch 151
Validation auc = 0.679343
Epoch 152
Validation auc = 0.666197
Epoch 153
Validation auc = 0.663850
Epoch 154
Loss = 2.7552e-02, PNorm = 59.7925, GNorm = 1.3602, lr_0 = 7.7759e-04
Validation auc = 0.669014
Epoch 155
Validation auc = 0.671362
Epoch 156
Validation auc = 0.671831
Epoch 157
Validation auc = 0.676526
Epoch 158
Validation auc = 0.673239
Epoch 159
Loss = 1.5349e-02, PNorm = 59.8718, GNorm = 0.4399, lr_0 = 7.7095e-04
Validation auc = 0.667136
Epoch 160
Validation auc = 0.671831
Epoch 161
Validation auc = 0.676056
Epoch 162
Validation auc = 0.669014
Epoch 163
Validation auc = 0.666197
Epoch 164
Loss = 2.5634e-02, PNorm = 59.9421, GNorm = 1.1916, lr_0 = 7.6436e-04
Validation auc = 0.671362
Epoch 165
Validation auc = 0.677465
Epoch 166
Validation auc = 0.688263
Epoch 167
Validation auc = 0.687324
Epoch 168
Validation auc = 0.683568
Epoch 169
Loss = 9.4019e-03, PNorm = 60.0206, GNorm = 0.3088, lr_0 = 7.5782e-04
Validation auc = 0.679812
Epoch 170
Validation auc = 0.675117
Epoch 171
Validation auc = 0.677465
Epoch 172
Validation auc = 0.673709
Epoch 173
Validation auc = 0.680282
Epoch 174
Loss = 5.3683e-03, PNorm = 60.0827, GNorm = 0.1637, lr_0 = 7.5134e-04
Validation auc = 0.698592
Epoch 175
Validation auc = 0.704225
Epoch 176
Validation auc = 0.704695
Epoch 177
Validation auc = 0.700939
Epoch 178
Validation auc = 0.698122
Epoch 179
Loss = 1.4180e-02, PNorm = 60.1466, GNorm = 0.1070, lr_0 = 7.4492e-04
Validation auc = 0.707512
Epoch 180
Validation auc = 0.695305
Epoch 181
Validation auc = 0.672300
Epoch 182
Validation auc = 0.670423
Epoch 183
Validation auc = 0.681690
Epoch 184
Loss = 7.3883e-02, PNorm = 60.2633, GNorm = 0.7482, lr_0 = 7.3855e-04
Validation auc = 0.667136
Epoch 185
Validation auc = 0.667606
Epoch 186
Validation auc = 0.687793
Epoch 187
Validation auc = 0.680282
Epoch 188
Validation auc = 0.663850
Epoch 189
Loss = 4.7825e-02, PNorm = 60.4202, GNorm = 0.2877, lr_0 = 7.3224e-04
Validation auc = 0.654460
Epoch 190
Validation auc = 0.652582
Epoch 191
Validation auc = 0.652113
Epoch 192
Validation auc = 0.648826
Epoch 193
Validation auc = 0.651643
Epoch 194
Loss = 3.4302e-02, PNorm = 60.5449, GNorm = 0.4786, lr_0 = 7.2598e-04
Validation auc = 0.657277
Epoch 195
Validation auc = 0.662911
Epoch 196
Validation auc = 0.660094
Epoch 197
Validation auc = 0.661502
Epoch 198
Validation auc = 0.659155
Epoch 199
Loss = 1.8543e-02, PNorm = 60.6342, GNorm = 0.3763, lr_0 = 7.1977e-04
Validation auc = 0.671362
Epoch 200
Validation auc = 0.680282
Epoch 201
Validation auc = 0.687324
Epoch 202
Validation auc = 0.682629
Epoch 203
Validation auc = 0.675587
Epoch 204
Loss = 1.3369e-03, PNorm = 60.6989, GNorm = 0.0178, lr_0 = 7.1362e-04
Validation auc = 0.667606
Epoch 205
Validation auc = 0.664789
Epoch 206
Validation auc = 0.667606
Epoch 207
Validation auc = 0.669953
Epoch 208
Validation auc = 0.668545
Epoch 209
Loss = 3.0691e-03, PNorm = 60.7459, GNorm = 0.1250, lr_0 = 7.0752e-04
Validation auc = 0.662911
Epoch 210
Validation auc = 0.667136
Epoch 211
Validation auc = 0.674648
Epoch 212
Validation auc = 0.676526
Epoch 213
Validation auc = 0.675117
Epoch 214
Loss = 1.3980e-02, PNorm = 60.7850, GNorm = 0.9013, lr_0 = 7.0147e-04
Validation auc = 0.669953
Epoch 215
Validation auc = 0.658216
Epoch 216
Validation auc = 0.663380
Epoch 217
Validation auc = 0.675587
Epoch 218
Validation auc = 0.681221
Epoch 219
Loss = 6.1905e-03, PNorm = 60.8306, GNorm = 0.1878, lr_0 = 6.9548e-04
Validation auc = 0.682629
Epoch 220
Validation auc = 0.678404
Epoch 221
Validation auc = 0.673709
Epoch 222
Validation auc = 0.677934
Epoch 223
Validation auc = 0.682629
Epoch 224
Loss = 6.8422e-03, PNorm = 60.8791, GNorm = 0.0816, lr_0 = 6.8953e-04
Validation auc = 0.688732
Epoch 225
Validation auc = 0.685915
Epoch 226
Validation auc = 0.678404
Epoch 227
Validation auc = 0.673239
Epoch 228
Validation auc = 0.670423
Epoch 229
Loss = 1.9235e-03, PNorm = 60.9303, GNorm = 0.0699, lr_0 = 6.8364e-04
Validation auc = 0.669484
Epoch 230
Validation auc = 0.680282
Epoch 231
Validation auc = 0.686385
Epoch 232
Validation auc = 0.674178
Epoch 233
Validation auc = 0.668545
Epoch 234
Loss = 1.0474e-03, PNorm = 60.9782, GNorm = 0.0116, lr_0 = 6.7779e-04
Validation auc = 0.664789
Epoch 235
Validation auc = 0.667606
Epoch 236
Validation auc = 0.668075
Epoch 237
Validation auc = 0.667606
Epoch 238
Validation auc = 0.666667
Epoch 239
Loss = 4.9621e-03, PNorm = 61.0255, GNorm = 0.1727, lr_0 = 6.7200e-04
Validation auc = 0.664319
Epoch 240
Validation auc = 0.664319
Epoch 241
Validation auc = 0.664319
Epoch 242
Validation auc = 0.662441
Epoch 243
Validation auc = 0.662441
Epoch 244
Loss = 3.9181e-03, PNorm = 61.0623, GNorm = 0.0621, lr_0 = 6.6625e-04
Validation auc = 0.663850
Epoch 245
Validation auc = 0.667606
Epoch 246
Validation auc = 0.666667
Epoch 247
Validation auc = 0.665258
Epoch 248
Validation auc = 0.665728
Epoch 249
Loss = 1.3105e-03, PNorm = 61.0851, GNorm = 0.0431, lr_0 = 6.6056e-04
Validation auc = 0.669014
Epoch 250
Validation auc = 0.668075
Epoch 251
Validation auc = 0.667136
Epoch 252
Validation auc = 0.673239
Epoch 253
Validation auc = 0.673709
Epoch 254
Loss = 3.3928e-03, PNorm = 61.1007, GNorm = 0.0225, lr_0 = 6.5491e-04
Validation auc = 0.674648
Epoch 255
Validation auc = 0.674648
Epoch 256
Validation auc = 0.671362
Epoch 257
Validation auc = 0.668545
Epoch 258
Validation auc = 0.673709
Epoch 259
Loss = 7.8129e-04, PNorm = 61.1150, GNorm = 0.0245, lr_0 = 6.4931e-04
Validation auc = 0.677934
Epoch 260
Validation auc = 0.682629
Epoch 261
Validation auc = 0.683099
Epoch 262
Validation auc = 0.684507
Epoch 263
Validation auc = 0.682629
Epoch 264
Loss = 2.2538e-03, PNorm = 61.1309, GNorm = 0.1409, lr_0 = 6.4376e-04
Validation auc = 0.685446
Epoch 265
Validation auc = 0.686385
Epoch 266
Validation auc = 0.686385
Epoch 267
Validation auc = 0.686385
Epoch 268
Validation auc = 0.686385
Epoch 269
Loss = 1.3278e-03, PNorm = 61.1428, GNorm = 0.1472, lr_0 = 6.3826e-04
Validation auc = 0.686854
Epoch 270
Validation auc = 0.686854
Epoch 271
Validation auc = 0.685915
Epoch 272
Validation auc = 0.690141
Epoch 273
Validation auc = 0.692958
Epoch 274
Loss = 1.2878e-03, PNorm = 61.1562, GNorm = 0.0215, lr_0 = 6.3280e-04
Validation auc = 0.689671
Epoch 275
Validation auc = 0.688263
Epoch 276
Validation auc = 0.686385
Epoch 277
Validation auc = 0.683568
Epoch 278
Validation auc = 0.680282
Epoch 279
Loss = 1.5730e-03, PNorm = 61.1665, GNorm = 0.1526, lr_0 = 6.2739e-04
Validation auc = 0.678873
Epoch 280
Validation auc = 0.680751
Epoch 281
Validation auc = 0.682629
Epoch 282
Validation auc = 0.689671
Epoch 283
Validation auc = 0.693897
Epoch 284
Loss = 3.3027e-03, PNorm = 61.1805, GNorm = 0.1853, lr_0 = 6.2203e-04
Validation auc = 0.694366
Epoch 285
Validation auc = 0.695305
Epoch 286
Validation auc = 0.691549
Epoch 287
Validation auc = 0.687793
Epoch 288
Validation auc = 0.685915
Epoch 289
Loss = 2.5711e-04, PNorm = 61.1920, GNorm = 0.0116, lr_0 = 6.1671e-04
Validation auc = 0.684038
Epoch 290
Validation auc = 0.681690
Epoch 291
Validation auc = 0.681221
Epoch 292
Validation auc = 0.679812
Epoch 293
Validation auc = 0.678873
Epoch 294
Loss = 5.3603e-03, PNorm = 61.2037, GNorm = 0.0082, lr_0 = 6.1144e-04
Validation auc = 0.686854
Epoch 295
Validation auc = 0.693427
Epoch 296
Validation auc = 0.690610
Epoch 297
Validation auc = 0.687324
Epoch 298
Validation auc = 0.679343
Epoch 299
Loss = 2.4725e-04, PNorm = 61.2197, GNorm = 0.0232, lr_0 = 6.0621e-04
Validation auc = 0.674648
Model 0 best validation auc = 0.739906 on epoch 33
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.894642
Ensemble test auc = 0.894642
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1528, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,829
Moving model to cuda
Epoch 0
Validation auc = 0.456338
Epoch 1
Validation auc = 0.476526
Epoch 2
Validation auc = 0.519718
Epoch 3
Validation auc = 0.569953
Epoch 4
Loss = 6.4532e-01, PNorm = 54.1583, GNorm = 1.3779, lr_0 = 6.5000e-04
Validation auc = 0.588732
Epoch 5
Validation auc = 0.658685
Epoch 6
Validation auc = 0.640376
Epoch 7
Validation auc = 0.684038
Epoch 8
Validation auc = 0.670892
Epoch 9
Loss = 5.5189e-01, PNorm = 54.3009, GNorm = 1.2908, lr_0 = 9.9743e-04
Validation auc = 0.661502
Epoch 10
Validation auc = 0.661502
Epoch 11
Validation auc = 0.673709
Epoch 12
Validation auc = 0.696244
Epoch 13
Validation auc = 0.698122
Epoch 14
Loss = 4.9947e-01, PNorm = 54.4967, GNorm = 1.9764, lr_0 = 9.8890e-04
Validation auc = 0.703756
Epoch 15
Validation auc = 0.707042
Epoch 16
Validation auc = 0.693897
Epoch 17
Validation auc = 0.692958
Epoch 18
Validation auc = 0.686854
Epoch 19
Loss = 4.5313e-01, PNorm = 54.6623, GNorm = 0.8146, lr_0 = 9.8045e-04
Validation auc = 0.690610
Epoch 20
Validation auc = 0.696244
Epoch 21
Validation auc = 0.706573
Epoch 22
Validation auc = 0.712676
Epoch 23
Validation auc = 0.723474
Epoch 24
Loss = 4.1464e-01, PNorm = 54.8526, GNorm = 1.0240, lr_0 = 9.7207e-04
Validation auc = 0.727230
Epoch 25
Validation auc = 0.722066
Epoch 26
Validation auc = 0.730516
Epoch 27
Validation auc = 0.736620
Epoch 28
Validation auc = 0.730047
Epoch 29
Loss = 3.8282e-01, PNorm = 55.0330, GNorm = 0.8163, lr_0 = 9.6376e-04
Validation auc = 0.714554
Epoch 30
Validation auc = 0.714554
Epoch 31
Validation auc = 0.721127
Epoch 32
Validation auc = 0.751174
Epoch 33
Validation auc = 0.740845
Epoch 34
Loss = 3.0459e-01, PNorm = 55.2424, GNorm = 0.9983, lr_0 = 9.5552e-04
Validation auc = 0.744131
Epoch 35
Validation auc = 0.718779
Epoch 36
Validation auc = 0.708451
Epoch 37
Validation auc = 0.710329
Epoch 38
Validation auc = 0.716432
Epoch 39
Loss = 3.8996e-01, PNorm = 55.4372, GNorm = 1.0682, lr_0 = 9.4735e-04
Validation auc = 0.729108
Epoch 40
Validation auc = 0.728169
Epoch 41
Validation auc = 0.725352
Epoch 42
Validation auc = 0.721127
Epoch 43
Validation auc = 0.723944
Epoch 44
Loss = 3.1085e-01, PNorm = 55.6732, GNorm = 1.1013, lr_0 = 9.3925e-04
Validation auc = 0.721127
Epoch 45
Validation auc = 0.704225
Epoch 46
Validation auc = 0.709859
Epoch 47
Validation auc = 0.688732
Epoch 48
Validation auc = 0.704225
Epoch 49
Loss = 3.3657e-01, PNorm = 55.9266, GNorm = 3.3387, lr_0 = 9.3122e-04
Validation auc = 0.707981
Epoch 50
Validation auc = 0.702347
Epoch 51
Validation auc = 0.712676
Epoch 52
Validation auc = 0.712207
Epoch 53
Validation auc = 0.702347
Epoch 54
Loss = 2.0302e-01, PNorm = 56.1837, GNorm = 1.2654, lr_0 = 9.2326e-04
Validation auc = 0.693427
Epoch 55
Validation auc = 0.706573
Epoch 56
Validation auc = 0.700000
Epoch 57
Validation auc = 0.706103
Epoch 58
Validation auc = 0.717840
Epoch 59
Loss = 1.7011e-01, PNorm = 56.4276, GNorm = 0.6831, lr_0 = 9.1537e-04
Validation auc = 0.690610
Epoch 60
Validation auc = 0.693897
Epoch 61
Validation auc = 0.702347
Epoch 62
Validation auc = 0.701878
Epoch 63
Validation auc = 0.723005
Epoch 64
Loss = 1.3914e-01, PNorm = 56.6771, GNorm = 0.7106, lr_0 = 9.0754e-04
Validation auc = 0.706103
Epoch 65
Validation auc = 0.720188
Epoch 66
Validation auc = 0.701878
Epoch 67
Validation auc = 0.719718
Epoch 68
Validation auc = 0.717840
Epoch 69
Loss = 1.1780e-01, PNorm = 56.8973, GNorm = 0.6514, lr_0 = 8.9978e-04
Validation auc = 0.706103
Epoch 70
Validation auc = 0.693427
Epoch 71
Validation auc = 0.697653
Epoch 72
Validation auc = 0.694836
Epoch 73
Validation auc = 0.686385
Epoch 74
Loss = 1.5352e-01, PNorm = 57.1340, GNorm = 2.4062, lr_0 = 8.9209e-04
Validation auc = 0.689202
Epoch 75
Validation auc = 0.728169
Epoch 76
Validation auc = 0.685915
Epoch 77
Validation auc = 0.666197
Epoch 78
Validation auc = 0.665258
Epoch 79
Loss = 1.7116e-01, PNorm = 57.3680, GNorm = 1.2961, lr_0 = 8.8447e-04
Validation auc = 0.671362
Epoch 80
Validation auc = 0.674648
Epoch 81
Validation auc = 0.685446
Epoch 82
Validation auc = 0.708920
Epoch 83
Validation auc = 0.701878
Epoch 84
Loss = 9.6569e-02, PNorm = 57.5935, GNorm = 0.8045, lr_0 = 8.7691e-04
Validation auc = 0.675587
Epoch 85
Validation auc = 0.682629
Epoch 86
Validation auc = 0.696714
Epoch 87
Validation auc = 0.690141
Epoch 88
Validation auc = 0.700000
Epoch 89
Loss = 1.5358e-01, PNorm = 57.8042, GNorm = 0.9092, lr_0 = 8.6941e-04
Validation auc = 0.708920
Epoch 90
Validation auc = 0.710798
Epoch 91
Validation auc = 0.697183
Epoch 92
Validation auc = 0.684038
Epoch 93
Validation auc = 0.671831
Epoch 94
Loss = 4.4489e-02, PNorm = 57.9794, GNorm = 1.0828, lr_0 = 8.6198e-04
Validation auc = 0.682629
Epoch 95
Validation auc = 0.710329
Epoch 96
Validation auc = 0.666667
Epoch 97
Validation auc = 0.686854
Epoch 98
Validation auc = 0.739437
Epoch 99
Loss = 9.0038e-02, PNorm = 58.1614, GNorm = 1.1425, lr_0 = 8.5461e-04
Validation auc = 0.721127
Epoch 100
Validation auc = 0.706573
Epoch 101
Validation auc = 0.697183
Epoch 102
Validation auc = 0.709390
Epoch 103
Validation auc = 0.709390
Epoch 104
Loss = 1.4181e-01, PNorm = 58.3390, GNorm = 1.0343, lr_0 = 8.4730e-04
Validation auc = 0.696244
Epoch 105
Validation auc = 0.707981
Epoch 106
Validation auc = 0.717840
Epoch 107
Validation auc = 0.720657
Epoch 108
Validation auc = 0.713146
Epoch 109
Loss = 5.0166e-02, PNorm = 58.5094, GNorm = 0.4474, lr_0 = 8.4006e-04
Validation auc = 0.722066
Epoch 110
Validation auc = 0.717371
Epoch 111
Validation auc = 0.701878
Epoch 112
Validation auc = 0.691080
Epoch 113
Validation auc = 0.698122
Epoch 114
Loss = 5.3577e-02, PNorm = 58.6726, GNorm = 0.3768, lr_0 = 8.3288e-04
Validation auc = 0.707512
Epoch 115
Validation auc = 0.715962
Epoch 116
Validation auc = 0.717371
Epoch 117
Validation auc = 0.709859
Epoch 118
Validation auc = 0.710329
Epoch 119
Loss = 2.1132e-02, PNorm = 58.7986, GNorm = 0.8050, lr_0 = 8.2576e-04
Validation auc = 0.707512
Epoch 120
Validation auc = 0.704695
Epoch 121
Validation auc = 0.701408
Epoch 122
Validation auc = 0.705634
Epoch 123
Validation auc = 0.706103
Epoch 124
Loss = 3.2450e-02, PNorm = 58.8916, GNorm = 0.8614, lr_0 = 8.1870e-04
Validation auc = 0.687324
Epoch 125
Validation auc = 0.684507
Epoch 126
Validation auc = 0.697183
Epoch 127
Validation auc = 0.685915
Epoch 128
Validation auc = 0.686854
Epoch 129
Loss = 3.5667e-02, PNorm = 58.9947, GNorm = 1.3946, lr_0 = 8.1170e-04
Validation auc = 0.700469
Epoch 130
Validation auc = 0.723944
Epoch 131
Validation auc = 0.723005
Epoch 132
Validation auc = 0.706103
Epoch 133
Validation auc = 0.685446
Epoch 134
Loss = 8.1061e-02, PNorm = 59.1094, GNorm = 2.0925, lr_0 = 8.0476e-04
Validation auc = 0.680751
Epoch 135
Validation auc = 0.684038
Epoch 136
Validation auc = 0.684038
Epoch 137
Validation auc = 0.678873
Epoch 138
Validation auc = 0.680282
Epoch 139
Loss = 5.0302e-02, PNorm = 59.2412, GNorm = 0.7818, lr_0 = 7.9788e-04
Validation auc = 0.683568
Epoch 140
Validation auc = 0.689202
Epoch 141
Validation auc = 0.691549
Epoch 142
Validation auc = 0.684977
Epoch 143
Validation auc = 0.669014
Epoch 144
Loss = 2.5570e-02, PNorm = 59.3464, GNorm = 0.5381, lr_0 = 7.9106e-04
Validation auc = 0.663850
Epoch 145
Validation auc = 0.661502
Epoch 146
Validation auc = 0.672300
Epoch 147
Validation auc = 0.678404
Epoch 148
Validation auc = 0.680751
Epoch 149
Loss = 3.0524e-02, PNorm = 59.4607, GNorm = 0.4821, lr_0 = 7.8430e-04
Validation auc = 0.675587
Epoch 150
Validation auc = 0.676526
Epoch 151
Validation auc = 0.683099
Epoch 152
Validation auc = 0.683099
Epoch 153
Validation auc = 0.681690
Epoch 154
Loss = 1.4087e-02, PNorm = 59.5498, GNorm = 0.6460, lr_0 = 7.7759e-04
Validation auc = 0.676995
Epoch 155
Validation auc = 0.675587
Epoch 156
Validation auc = 0.681690
Epoch 157
Validation auc = 0.681690
Epoch 158
Validation auc = 0.680751
Epoch 159
Loss = 6.6049e-03, PNorm = 59.6086, GNorm = 0.1813, lr_0 = 7.7095e-04
Validation auc = 0.683568
Epoch 160
Validation auc = 0.692958
Epoch 161
Validation auc = 0.695775
Epoch 162
Validation auc = 0.694836
Epoch 163
Validation auc = 0.694366
Epoch 164
Loss = 3.9837e-03, PNorm = 59.6630, GNorm = 0.0900, lr_0 = 7.6436e-04
Validation auc = 0.691549
Epoch 165
Validation auc = 0.687793
Epoch 166
Validation auc = 0.689202
Epoch 167
Validation auc = 0.690610
Epoch 168
Validation auc = 0.690141
Epoch 169
Loss = 4.2392e-03, PNorm = 59.7126, GNorm = 0.1660, lr_0 = 7.5782e-04
Validation auc = 0.687793
Epoch 170
Validation auc = 0.686854
Epoch 171
Validation auc = 0.693427
Epoch 172
Validation auc = 0.687793
Epoch 173
Validation auc = 0.684977
Epoch 174
Loss = 5.8125e-03, PNorm = 59.7646, GNorm = 0.2011, lr_0 = 7.5134e-04
Validation auc = 0.684038
Epoch 175
Validation auc = 0.687324
Epoch 176
Validation auc = 0.686385
Epoch 177
Validation auc = 0.698592
Epoch 178
Validation auc = 0.701408
Epoch 179
Loss = 2.0040e-02, PNorm = 59.8268, GNorm = 1.0643, lr_0 = 7.4492e-04
Validation auc = 0.684977
Epoch 180
Validation auc = 0.678404
Epoch 181
Validation auc = 0.681690
Epoch 182
Validation auc = 0.682629
Epoch 183
Validation auc = 0.682629
Epoch 184
Loss = 1.0306e-02, PNorm = 59.9042, GNorm = 0.0722, lr_0 = 7.3855e-04
Validation auc = 0.683099
Epoch 185
Validation auc = 0.683568
Epoch 186
Validation auc = 0.686385
Epoch 187
Validation auc = 0.691080
Epoch 188
Validation auc = 0.698592
Epoch 189
Loss = 8.8574e-03, PNorm = 59.9780, GNorm = 0.4393, lr_0 = 7.3224e-04
Validation auc = 0.698592
Epoch 190
Validation auc = 0.689202
Epoch 191
Validation auc = 0.679812
Epoch 192
Validation auc = 0.679812
Epoch 193
Validation auc = 0.690141
Epoch 194
Loss = 6.5732e-03, PNorm = 60.0414, GNorm = 0.5211, lr_0 = 7.2598e-04
Validation auc = 0.692488
Epoch 195
Validation auc = 0.687793
Epoch 196
Validation auc = 0.683568
Epoch 197
Validation auc = 0.678873
Epoch 198
Validation auc = 0.682160
Epoch 199
Loss = 1.2934e-03, PNorm = 60.0885, GNorm = 0.0651, lr_0 = 7.1977e-04
Validation auc = 0.689202
Epoch 200
Validation auc = 0.697653
Epoch 201
Validation auc = 0.709859
Epoch 202
Validation auc = 0.723944
Epoch 203
Validation auc = 0.720657
Epoch 204
Loss = 4.4593e-02, PNorm = 60.1459, GNorm = 0.7845, lr_0 = 7.1362e-04
Validation auc = 0.724883
Epoch 205
Validation auc = 0.714085
Epoch 206
Validation auc = 0.714085
Epoch 207
Validation auc = 0.703756
Epoch 208
Validation auc = 0.684977
Epoch 209
Loss = 9.2974e-03, PNorm = 60.2220, GNorm = 0.4531, lr_0 = 7.0752e-04
Validation auc = 0.669484
Epoch 210
Validation auc = 0.669953
Epoch 211
Validation auc = 0.694836
Epoch 212
Validation auc = 0.707042
Epoch 213
Validation auc = 0.706573
Epoch 214
Loss = 5.6451e-02, PNorm = 60.3281, GNorm = 0.6947, lr_0 = 7.0147e-04
Validation auc = 0.705634
Epoch 215
Validation auc = 0.700469
Epoch 216
Validation auc = 0.710329
Epoch 217
Validation auc = 0.700000
Epoch 218
Validation auc = 0.690610
Epoch 219
Loss = 8.4649e-03, PNorm = 60.4580, GNorm = 0.0409, lr_0 = 6.9548e-04
Validation auc = 0.682629
Epoch 220
Validation auc = 0.680282
Epoch 221
Validation auc = 0.678404
Epoch 222
Validation auc = 0.685915
Epoch 223
Validation auc = 0.684507
Epoch 224
Loss = 2.0123e-02, PNorm = 60.5529, GNorm = 0.0253, lr_0 = 6.8953e-04
Validation auc = 0.684038
Epoch 225
Validation auc = 0.686385
Epoch 226
Validation auc = 0.688263
Epoch 227
Validation auc = 0.687324
Epoch 228
Validation auc = 0.685915
Epoch 229
Loss = 1.1302e-02, PNorm = 60.6530, GNorm = 0.4826, lr_0 = 6.8364e-04
Validation auc = 0.692019
Epoch 230
Validation auc = 0.697183
Epoch 231
Validation auc = 0.702347
Epoch 232
Validation auc = 0.702347
Epoch 233
Validation auc = 0.703756
Epoch 234
Loss = 5.8444e-03, PNorm = 60.7462, GNorm = 0.3346, lr_0 = 6.7779e-04
Validation auc = 0.705164
Epoch 235
Validation auc = 0.702817
Epoch 236
Validation auc = 0.699061
Epoch 237
Validation auc = 0.692958
Epoch 238
Validation auc = 0.689202
Epoch 239
Loss = 2.9429e-03, PNorm = 60.8088, GNorm = 0.2211, lr_0 = 6.7200e-04
Validation auc = 0.684977
Epoch 240
Validation auc = 0.687793
Epoch 241
Validation auc = 0.692958
Epoch 242
Validation auc = 0.693427
Epoch 243
Validation auc = 0.693897
Epoch 244
Loss = 2.2604e-03, PNorm = 60.8497, GNorm = 0.0505, lr_0 = 6.6625e-04
Validation auc = 0.693897
Epoch 245
Validation auc = 0.693897
Epoch 246
Validation auc = 0.694366
Epoch 247
Validation auc = 0.697653
Epoch 248
Validation auc = 0.704695
Epoch 249
Loss = 3.7389e-03, PNorm = 60.8826, GNorm = 0.1111, lr_0 = 6.6056e-04
Validation auc = 0.708920
Epoch 250
Validation auc = 0.707512
Epoch 251
Validation auc = 0.702347
Epoch 252
Validation auc = 0.698592
Epoch 253
Validation auc = 0.699531
Epoch 254
Loss = 1.1213e-02, PNorm = 60.9280, GNorm = 0.0089, lr_0 = 6.5491e-04
Validation auc = 0.706573
Epoch 255
Validation auc = 0.705634
Epoch 256
Validation auc = 0.699061
Epoch 257
Validation auc = 0.692958
Epoch 258
Validation auc = 0.684038
Epoch 259
Loss = 6.4642e-04, PNorm = 60.9764, GNorm = 0.0126, lr_0 = 6.4931e-04
Validation auc = 0.679343
Epoch 260
Validation auc = 0.672300
Epoch 261
Validation auc = 0.673239
Epoch 262
Validation auc = 0.675587
Epoch 263
Validation auc = 0.672300
Epoch 264
Loss = 2.2484e-03, PNorm = 61.0406, GNorm = 0.0343, lr_0 = 6.4376e-04
Validation auc = 0.672770
Epoch 265
Validation auc = 0.670423
Epoch 266
Validation auc = 0.667606
Epoch 267
Validation auc = 0.669484
Epoch 268
Validation auc = 0.667606
Epoch 269
Loss = 1.1317e-03, PNorm = 61.1005, GNorm = 0.0122, lr_0 = 6.3826e-04
Validation auc = 0.669953
Epoch 270
Validation auc = 0.670892
Epoch 271
Validation auc = 0.670892
Epoch 272
Validation auc = 0.670423
Epoch 273
Validation auc = 0.670423
Epoch 274
Loss = 1.3368e-03, PNorm = 61.1347, GNorm = 0.0077, lr_0 = 6.3280e-04
Validation auc = 0.670423
Epoch 275
Validation auc = 0.671362
Epoch 276
Validation auc = 0.674178
Epoch 277
Validation auc = 0.677934
Epoch 278
Validation auc = 0.676995
Epoch 279
Loss = 4.4056e-03, PNorm = 61.1656, GNorm = 0.1831, lr_0 = 6.2739e-04
Validation auc = 0.675117
Epoch 280
Validation auc = 0.676056
Epoch 281
Validation auc = 0.675117
Epoch 282
Validation auc = 0.674178
Epoch 283
Validation auc = 0.670423
Epoch 284
Loss = 6.8670e-03, PNorm = 61.1960, GNorm = 0.4553, lr_0 = 6.2203e-04
Validation auc = 0.672300
Epoch 285
Validation auc = 0.673239
Epoch 286
Validation auc = 0.678404
Epoch 287
Validation auc = 0.680751
Epoch 288
Validation auc = 0.690141
Epoch 289
Loss = 1.3692e-03, PNorm = 61.2419, GNorm = 0.1723, lr_0 = 6.1671e-04
Validation auc = 0.695305
Epoch 290
Validation auc = 0.696714
Epoch 291
Validation auc = 0.696714
Epoch 292
Validation auc = 0.696244
Epoch 293
Validation auc = 0.692488
Epoch 294
Loss = 1.1500e-02, PNorm = 61.3046, GNorm = 0.3851, lr_0 = 6.1144e-04
Validation auc = 0.684038
Epoch 295
Validation auc = 0.676995
Epoch 296
Validation auc = 0.671362
Epoch 297
Validation auc = 0.666197
Epoch 298
Validation auc = 0.665728
Epoch 299
Loss = 3.2830e-03, PNorm = 61.3612, GNorm = 0.0713, lr_0 = 6.0621e-04
Validation auc = 0.665728
Model 0 best validation auc = 0.751174 on epoch 32
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.881397
Ensemble test auc = 0.881397
10-fold cross validation
	Seed 0 ==> test auc = 0.859121
	Seed 1 ==> test auc = 0.907887
	Seed 2 ==> test auc = 0.899458
	Seed 3 ==> test auc = 0.904877
	Seed 4 ==> test auc = 0.901264
	Seed 5 ==> test auc = 0.854907
	Seed 6 ==> test auc = 0.838049
	Seed 7 ==> test auc = 0.893438
	Seed 8 ==> test auc = 0.894642
	Seed 9 ==> test auc = 0.881397
Overall test auc = 0.883504 +/- 0.023080
Elapsed time = 0:05:27
